Building A System of Holistic Accountability in Partner School Districts: District, School, and Instructional Data Teams

1
Building A System of Holistic Accountability in
Partner School Districts District, School, and
Instructional Data Teams

• Connecticut State Department of Education

2
Objectives

• 1) Review State Accountability Legislation,
  Public Act 08-153, Section 4
• 2) Understand what it means to be a Partner
  District, including what resources are available
  and what is required
• 3) Learn about Holistic and Reciprocal
  Accountability and
• 4) Understand the required accountability system
  (District, School, and Instructional Data Teams).

3
Legislation Public Act 08-153 Section 4

• (2) (A) Require an operations audit to identify
  possible programmatic savings and an
  instructional audit to identify any deficits in
  curriculum and instruction or in the learning
  environment of the school or district.
• (H) Direct the local or regional board of
  education for the school or district to develop
  and implement a plan addressing deficits in
  achievement and in the learning environment as
  recommended in the instructional audit.

4
Legislation, cont.

• (M) Require local and regional boards of
  education to (i) undergo training to improve
  their operational efficiency and effectiveness as
  leaders of their districts improvement plans.
• (ii) Submit an annual action plan to the
  Commissioner of Education outlining how, when,
  and in what manner their effectiveness shall be
  monitored.

5
Legislation, cont.

• (3) If a directive of the State Board of
  Education pursuant to subparagraph (C), (D), (E),
  or (L) of subdivision (2) of this subsection or a
  directive to implement a plan pursuant to
  subparagraph (H) of said subdivision affects
  working conditions, such directive shall be
  carried out in accordance with the provision of
  sections 10-153a to 10-153n, inclusive.

6
District System of Accountability

• Each district will be required to develop a
  system of accountability which includes the
  following components of the Connecticut
  Accountability for Learning Initiative (CALI)
• Focus on student achievement
• Prioritized standards-based curriculum
• Use of data to inform instruction
• Frequent assessment of student progress
• Use of research-based strategies
• Collaborative teams focused on student learning
  and
• All adults held accountable for student
  achievement.

7
District System of Accountability, cont.

• The accountability system must include
• District Data Team
• School Data Teams and
• Instructional Data Teams (grade level and/or
  content area teacher teams).

8
Relationship and Synergy Between Data Teams
District Data Team
School Data Team
School Data Team
School Data Team
School Data Team
Inst. Data Team
Inst. Data Team
Inst. Data Team
Inst Data Team
Inst. Data Team
Inst Data Team
Inst. Data Team
Inst. Data Team
Inst. Data Team
Inst. Data Team
Inst Data Team
Inst. Data Team
Inst. Data Team
Inst. Data Team
Inst. Data team
Inst. Data Team
Inst Data Team
Inst. Data Team
Inst. Data Team
Inst. Data Team
Inst. Data Team
Inst. Data Team
9
Partner Districts Identification Criteria

• Partner Districts are those districts that have
  been identified in Year 3 or beyond of being in
  need of improvement at the whole district level
  in reading, math or both.

10
Partner District Supports

• Are provided technical assistance teams from the
  Connecticut State Department of Education
• Can access CALI training at no cost
• Receive funding for two Demonstration Schools
• Receive on-site consultant support from
  Leadership and Learning Center
• Have access to coaches for schools

11
Partner District Requirements

• Implement District, School, and Instructional
  Data Teams
• Are assessed at the district and school level by
  Cambridge Education
• Revise District Improvement Plans for approval by
  State Board of Education to include components of
  CALI and respond to Cambridge Assessment findings

12
Connecticut Accountability for Learning
Initiative
13
CALI Professional Development includes

• FOR ALL EDUCATORS
• Data-Driven Decision Making/Data Teams (DDDM/DT)
• Making Standards Work (MSW)
• Effective Teaching Strategies (ETS)
• Common Formative Assessments (CFA)
• Improving School Climate (ISC)
• Scientific Research Based Interventions (SRBI,
  also known as Response to Intervention)
• Register at www.sdecali.net

• FOR COACHES LEADERS
• Coaching Instructional Data Teams
• Coaching Effective Teaching Strategies
• The Change Academy Leading Change Getting
  Everyone on Board
• Classroom Data Feedback, Follow up Follow
  Through
• School Improvement Planning No Child Left
  Behind
• FOR PARAPROFESSIONALS
• CALI Overview

14
Levels of Training

• Basic training provides foundational information
• Certification training allows participants to
  turnkey basic training in a trainer of trainers
  model (completing basic training is a
  pre-requisite)
• Certification is offered in DDDM/DT, MSW, ETS,
  CFA, ISC and SRBI

15
Why We Have to Work Differently

• Old Rules
• Some students will learn it and some wont.
• If they dont learn it, its their fault.

• New Rules
• All students will learn it.
• If they dont learn it, the professionals have
  to take responsibility.

16
Holistic Accountability

• The concept of holistic accountability includes
  three foundational components
• No child will be held more accountable than the
  adults in the system
• Tier II Indicators (adult actions that are cause
  data), must be as carefully monitored as Tier I
  Indicators (student achievement outcomes that are
  effect data) and
• Adult cause and student effect data must be
  monitored on every level of the system through
  District Data Teams, School Data Teams, and
  Instructional Data Teams.

17
CSDE Beliefs

• Improving schools and districts must be a
  transparent process where honest bad news is
  confronted.
• The purpose of improvement planning is to improve
  student achievement. It is not to sort,
  categorize, or label districts, schools, staff,
  or students.

18
CSDE Beliefs, cont.

• The actions of the adults in the system have the
  power to improve student achievement, regardless
  of the challenges.
• District and School Improvement Plans are more
  than just a compliance exercise.
• District and School Improvement Plans are most
  effective when the scope of the goals and actions
  are limited, and implementation and effectiveness
  of plan are constantly monitored to allow for
  revisions as needed.

19
Leadership Learning Matrix
Effects/Results (stud.out.) Lucky High results, low understanding of antecedents Replication of success unlikely Leading High results, high understanding of antecedents Replication of success likely
Effects/Results (stud.out.) Losing Low results, low understanding of antecedents Replication of failure likely Learning Low results, high understanding of antecedents Replication of mistakes unlikely
Antecedents/Cause Data(adult actions) Antecedents/Cause Data(adult actions) Antecedents/Cause Data(adult actions)
(Reeves, 2006)
20
Reciprocal Accountability

• If the district (or state) is to hold schools
  accountable for producing specific outcomes for
  their students, the district (or state) has the
  responsibility to provide those schools with the
  resources (human, material, and intellectual) and
  the conditions necessary to produce those
  outcomes.
• - Richard Elmore
• Hess, F. M. (2006). Urban school reform
  Lessons from San Diego. Cambridge, MA Harvard
  Education Press.

21
Data Teams

• Data Teams are groups of professional educators
  working collaboratively to analyze the effect of
  their actions on identified student outcomes.
• Data teams adhere to continuous improvement
  cycles, examine patterns and trends, and
  establish specific timelines, roles, and
  responsibilities to facilitate analysis that
  results in action.
• White, S. (2005). Beyond the numbers.
  Englewood, CO Advanced Learning Press.

22
Data Team Process(applies to District, School,
and Instructional Data Teams)

• Step 1 Collect and chart data (cause and
  effect)
• Step 2 Analyze strengths and obstacles
• Step 3 Establish SMART (specific, measurable,
  achievable, relevant, time-bound) goals set,
  review, revise
• Step 4 Select adult actions/instructional
  strategies
• Step 5 Determine Results Indicators to
  determine 1) if strategies are being implemented,
  and 2) if they are having the desired impact on
  student achievement

23
Suggested District Data Team (DDT) Composition

• Superintendent
• Assistant Superintendent(s)
• Curriculum Director
• Content Area Directors/Supervisors
• SPED Director
• ELL Director

• Principal Representation
• Teacher Representation
• Board of Education Representation
• Union Representation
• Parent Representation

24
District Data Teams

• Responsibilities
• Meet every 4 to 8 weeks
• Develop District Improvement Plan (DIP)
• Approve School Improvement Plans (SIP) and
  monitor School Data Teams
• Monitor DIP implementation and effectiveness, and
  revise accordingly
• Report on district progress to all stakeholders

25
Tier I Indicators

• Tier I Indicator specific indicators used to
  determine whether state and district learning
  expectations have been achieved as evidenced by
  student effect data (e.g., The percentage of
  Latino students that are proficient in estimation
  will increase from 57 percent to 75 percent as
  measured by the spring 2010 Connecticut Mastery
  Test).

26
Tier II Indicators

• Tier II Indicator specific actions taken by
  adults to cause student achievement outcomes
  (e.g., One hundred percent of faculty, student
  support staff, and administrators will
  participate in the Data Team process).

27
DDT Develop DIP

• DDT decides the Tier I Indicators, which are
  district goals based upon analysis of district
  data (no more than 3-5).
• Ex. By 2011 the of students reading at the
  Proficient level or greater on Grade 4 CMT will
  be at least 75.

• Step One
• Deciding
• Upon
• Goals

28
DDT Develop DIP, cont.

• Factors to consider when developing Tier I
  Indicators
• What does the data say about past performance
  patterns and trends?
• How much time do I have?
• How many resources am I going to devote?
• How many other things do I have to do?
• Initially, make Tier I Indicators tentative and
  revisit them after drafting plan to determine if
  they still make sense.

29
SAMPLE DIP
Goal Statement Historical Data Historical Data Historical Data Historical Data Historical Data Historical Data Goal
of students in Grade 4 Proficient or above in 05-06 06-07 07-08 08-09 09-10 2011
Reading 49.4 45.3 47.4 54.5
Math 64.9 66.7 69.4 79.8
Writing 76 73.1 74.6 85.8

Number of days of school suspensions 12,000 13,000 11,000 9,350
30
DDT Develop DIP, cont.

• Step Two
• Deciding
• Upon
• Adult
• Actions
• (Tier IIs)

• The District decides on a limited number (3-5) of
  Tier II Indicators.
• There are many things the District does that are
  not included in the Plan - these are considered
  Business As Usual.
• District Tier IIs will often effect Schools by
  giving them direction.
• Tier IIs should be quantified whenever possible.

31
DDT Develop DIP, cont.

• Step Two
• Deciding
• Upon
• Adult
• Actions
• (Tier IIs)

• Examples of District Tier IIs
• Improve the use of available time at all schools.
  
• Improve the quality of newly hired teachers.
• 100 of School Data Teams will all be functioning
  at proficient levels by Spring 2010.

32
DDT Develop DIP, cont.

• Step Three
• Writing
• An
• Action
• Plan

• For each Tier II it develops, a subcommittee of
  the District Data Team writes an Action Plan
  describing the major steps involved in carrying
  out the Tier II, including who is responsible,
  associated costs, and timelines.

33
DDT Develop DIP, cont.

• Step Four
• Creating
• Results
• Indicators
• (How will we know we are on track?)

• For each Tier II Indicator it creates, the DDT
  creates a set of Results Indicators.
• Results Indicators are data streams of adult
  actions and student outcomes that inform the team
  about implementation progress and impact on
  student achievement.
• Results Indicator data allow the DDT to determine
  when to revise, expand, or eliminate Tier II
  Indicators.

34
DDT Develop DIP, cont.

• Adult Action Results Indicators
• Are the adults doing what they said they would
  do?

• Student Outcome Results Indicators
• Are the students getting better at the critical
  skill(s) we identified?

35
DDT Develop DIP, cont.

• Adult Action Results Indicators Examples
• The of Data Team meetings held.
• The quality of Data Team meetings held.

• Student Outcome Results Indicators Examples
• of students improving performance on DRA.
• of student proficient or better on district
  benchmark assessment.

36
Sample District Action Plan
Tier II Indicator - Priority Adult Actions (Cause Data)
1. Fill all vacant positions with high quality teachers as early as possible and retain only those who demonstrate superior performance.
Strategies Implement early retirement announcement incentive of 1,000. Continue benefits for all teachers who retire before June 30 through the summer. Post all shortage positions by January. Create New Teacher Academy and require attendance.
Results Indicators Data on retirees, resignations, recruitment, hiring dates, openings. Comparison of student performance in low ability teacher class to high ability teacher class.
37
DDT Approve School Improvement Plans (SIP)
Monitor School Data Teams (SDT)

• The DDT, or a subgroup of it, should provide
  assistance to schools who are developing SIPs and
  ultimately approve the plans to make sure that
  they are operationalizing the DIP (SIPs must also
  be approved by local Board of Education).
• The DDT should monitor progress of SDTs
  throughout the year.

38
DDT Monitor DIP implementation and
effectiveness, and revise accordingly

• District Data Team
• Monitors the Results Indicators to determine if
  the Tier II Indicators are generating the desired
  results
• Changes the DIP in response to the data
  (Data-Driven Decision Making) and
• Expands, revises, or deletes Tier II Indicators
  and/or action steps as needed.

39
DDT Report on District Progress

• As public school employees, it is the DDTs
  responsibility to regularly report progress to
  the schools and the community.
• Reports must be written in a user-friendly
  format.
• Data Walls and District Data Showcases are
  effective ways to inform the public of progress.
• Keeping public informed can be helpful when the
  district needs to seek community support.

40
Data Wall
(Tier II) Strategies Actions of the adults
(Tier III) Analysis Why are we getting the
results we are?
(Tier I) Data State and district
41
Sample Data Walls

• Topic for professional conversations
• Located in prominent places

42
(No Transcript)
43
Suggested School Data Team (SDT) Composition

• Grade level teacher representation
• Content area teacher representation
• Special education representation
• Guidance Counselor/Social Worker
• Parent representation
• Union representation
• Administration

44
School Data Teams

• Responsibilities
• Meet on at least on a monthly basis
• Develop SIP
• Monitor Instructional Data Teams
• Monitor SIP implementation and effectiveness, and
  revise accordingly
• Report on district progress to all stakeholders

45
SDT Develop SIP

• Each School must address all of the Tier I
  Indicators in the DIP.
• Schools have the freedom to determine additional
  Tier I Indicators, but must keep in mind what the
  ultimate district goals are and where the school
  is starting from.
• Be able to defend your goal to the DDT.

• Step One
• Deciding
• Upon
• Goals

46
SDT Develop SIP, cont.

• Factors to consider when developing Tier I
  Indicators
• What does the data say about past performance
  patterns and trends?
• How much time do I have?
• How many resources am I going to devote?
• How many other things do I have to do?
• Initially, make Tier I Indicators tentative and
  revisit them after drafting plan to determine if
  they still make sense.

47
Kennedy Elementary School SIP
Goal Statement Historical Data Historical Data Historical Data Historical Data School Goal District Goal
of students in Grade 4 Proficient or above in 05-06 06-07 07-08 08-09 2011 2011
Reading 70 78 82 90 54.5
Math 65 70 72 85 79.8
Writing 85 82 88 89 85.8

Number of total days of school suspensions 150 165 200 35 less 15 less
48
Richards Elementary School SIP
Goal Statement Historical Data Historical Data Historical Data Historical Data School Goal District Goal
of students in Grade 4 Proficient or above in 05-06 06-07 07-08 08-09 2011 2011
Reading 25 24 30 50 54.5
Math 42 48 45 60 79.8
Writing 50 55 58 65 85.8

Number of total days of school suspensions 60 62 70 20 less 15 less
49
SDT Develop SIP, cont.

• Step Two
• Deciding
• Upon
• Adult
• Actions
• (Tier IIs)

• The School decides on a limited number (3-5) of
  Tier II Indicators.
• There are many things the School does that are
  not included in the Plan - these are considered
  Business As Usual.
• District Tier IIs will often effect Schools by
  giving them direction.
• Tier IIs should be quantified whenever possible.

50
SDT Develop SIP, cont.

• Step Two
• Deciding
• Upon
• Adult
• Actions
• (Tier IIs)

• Examples of School Tier IIs
• 100 of lesson plans will be aligned with State
  Grade Level Expectations, and District Curriculum
  by Spring 2009.
• 100 of all Instructional Data Teams will be
  functioning at proficient levels by Spring 2010.

51
SDT Develop SIP, cont.

• Step Three
• Writing
• An
• Action
• Plan

• For each Tier II Indicator it develops, the SDT
  writes an Action Plan describing the major steps
  involved in carrying out the Tier II, including
  who is responsible, associated costs, and
  timelines.

52
SDT Develop SIP, cont.

• Step Four
• Creating
• Results
• Indicators
• (How will we know we are on track?)

• For each Tier II it creates, the SDT creates a
  set of Results Indicators data streams of adult
  actions and student outcomes that inform the team
  about implementation progress and impact on
  student achievement.
• Results Indicator data allow the DDT to determine
  when to revise, expand, or eliminate Tier II
  Indicators.

53
SDT Develop SIP, cont.

• Adult Action Results Indicators Examples
• The of classrooms with student-friendly
  objectives posted.
• The of classrooms consistently implementing
  school-wide rules.

• Student Outcome Results Indicators Examples
• Increase in of students proficient or better on
  monthly school-wide assessment.
• Decrease in of students referred to the office
  for behavior infractions.

54
SDT Develop SIP, cont.

• Each SIP must be presented to the DDT.
• The presenting team must be able to defend the
  SIP.
• The DDT, or a subcommittee of it, must approve
  all SIPs.
• SIPs must also be approved by the local Board of
  Education.

55
SDT Monitor Instructional Data Teams

• School Data Teams
• Monitor the Results Indicators to determine if
  the Tier II Indicators are generating the desired
  results
• Change the SIP in response to the data
  (Data-Driven Decision Making) and
• Expand, revise, or delete Tier II Indicators
  and/or action steps as needed.

56
SDT Report on district progress to all
stakeholders

• As public school employees, it is the SDTs
  responsibility to regularly report progress to
  the school, district, and larger community.
• Reports must be written in a user-friendly
  format.
• Data Walls and a School Data Showcase are
  effective ways to inform the public of progress.
• Keeping public informed can be helpful when the
  school needs to seek community support.

57
Instructional Data Teams (IDT)

• Responsibilities
• Meet on a least a bi-monthly basis
• Develop and review results from Common Formative
  Assessments
• Analyze student work
• Establish SMART goals
• Agree upon Effective Teaching Strategies and
  interventions to implement before summative
  assessment is administered
• Some teams choose to collaboratively plan
  lessons

58
Instructional Data Teams Composition

• Grade level or content area teams that examine
  individual student work and data generated from
  common formative assessments.
• IDTs have collaborative, structured, scheduled
  meetings that focus on the effectiveness of
  teaching and learning.

59
Asking the Right Questions

• What does student achievement look like?
• What variables that affect student achievement
  are within your control?
• What adult actions in the school are impacting
  achievement results?
• How do you currently explain your results in
  student achievement?

60
Data Worth Collecting Have a Purpose

• How do you use data to inform instruction and
  improve student achievement?
• How do you triangulate cause (adult action) and
  effect (student achievement) data?
• In the absence of data, what is used as a basis
  for instructional decisions?

61
Vertical Data Team
62
Horizontal Data Team
63
Specialist Data Team
64
Frequency and Length of Data Team Meetings

• Varies Weekly to once a month
• Shortest (45 minutes) to longest (120 minutes)
• Schools that realize the greatest shift to a
  data culture scheduled meetings once a week!

65
Data Team Leader and Principal Debriefs

• IDT Leaders should be teachers, not
  administrators
• Meet at least monthly to discuss
• Achievement gaps
• Successes and challenges
• Progress monitoring
• Assessment schedules
• Intervention needs
• Resource needs

66
Questions? Comments?

• Meghan G. Martins, Education Consultant
• [phone]
• meghan.martins_at_ct.gov
• Michael Wasta, External Consultant
• michaelwasta_at_yahoo.com
• Kathleen Wedge, Education Consultant
• [phone]
• kathleen.wedge_at_ct.gov