Urban Education Challenges: Formative Assessment as Part of the Solution

1
Urban Education ChallengesFormative
Assessmentas Part of the Solution
2
(No Transcript)
3
Formative Assessment as Part of the Solution

• General goal Move Learning Raise Student
  Achievement
• Related goal Valid assessments for intended
  purpose(s)
• Different journeys toward the same end
• Making adjustments based on data
• Lessons learned along the way

4
Different journeys toward the same end

• Why?The motivation and rationale behind the
  assessments
• How?Approaches that fit each district
• What?What they are doing and what they are
  learning

5
Presenters

• Dr. Janet FilbinDirector of Assessment
  TeamJefferson County, CO School District

6
Presenters

• Dr. Janet FilbinDirector of Assessment
  TeamJefferson County, CO School District
• Dr. Joe JacovinoChief Accountability OfficerSt.
  Louis, MO Public Schools

7
Presenters

• Dr. Janet FilbinDirector of Assessment
  TeamJefferson County, CO School District
• Dr. Joe JacovinoChief Accountability OfficerSt.
  Louis, MO Public Schools
• Dr. Cleo FigguresDirector of Assessment, School
  District of Philadelphia, PA(New CAO for St.
  Louis, MO Public Schools)

8
Meeting the Urban Education Challenges in
Jeffco School District

• Dispelling The
• Urban Myth!

Janet Filbin Jeffco School District Golden,
Colorado jfilbin_at_jeffco.k12.co.us
9
JEFFCO PUBLIC SCHOOLS District Demographics

• Largest district in Colorado
• Approximately 85,000 students K-12
• 25 of students are identified as minority
• Notable shifts in demographics over the past 5
  years

10
Strategic Plan Objectives

• Ensure all students demonstrate achievement of
  academic content standards.
• Ensure all students with special needs
  demonstrate achievement of academic content
  standards and receive appropriate services and
  instruction.
• Ensure students and instructional personnel use
  curricula, assessments, and the tools of
  technology aligned with district content
  standards as the basis for the teaching and
  learning process.

11
Reality check please

• Student achievement was not what we wanted
• Reading scores were flat or up and down at most
  grade levels
• Math scores were increasing across most grade
  levels
• Increase in student mobility, students identified
  as Hispanic, and those receiving free and reduced
  lunch
• Very few standardized performance measures
  administered across the year to determine
  instructional impact

12
The Urban Myth

• The reason our scores are low is
• Kids in our school are more mobile
• Our student population is changing-we have more
  minorities
• We have more English language learners than ever
  before
• Our school is more impacted with students with
  disabilities

13
Dispelling the Urban Myth
MCREL Balanced Leadership
Secondary Reform
District Initiatives
Responsive Instruction, Intervention, and
Assessment (Jeffcos RTI Model)
Data-driven Decision-making
I2 (a) Pilot
14
The I2 (a) Pilot

• A combination of 2-3 predictive tests plus a CBM
• Intended to provide benchmark information about
  current progress toward grade level expectations
• Gives teachers a progress monitoring tool to
  evaluate alignment of core curriculum on a
  regular basis and effectiveness of interventions

15
Intended outcomes

• Administrators will..
• Use the data to have a more structured, objective
  method to monitor school, grade, or course
  implementation of the curriculum and
  instructional impact
• Identify the pattern of strengths and challenges
  in achievement to better plan for resources use

• Teachers will..
• Use the data to be more objective about student
  learning-no more cardiac tests!
• Be able to judge if their curriculum/instruction
  matches state standards
• Have a better picture of achievement gaps to
  provide more responsive interventions
• Continuously evaluate their instruction and
  interventions through progress monitoring

16
Not So Unintended Outcomes

• Curriculum development/alignment at the district
  level
• Collaboration between departments
• Increased use of data to drive school improvement
  decision

17
Lessons learned

• Vision foresight!
• The road to curriculum should not be less
  traveled.
• Just because you learned to ride a bike doesnt
  mean youre a master cyclist.
• Building the plane while flying requires quantum
  thinking!

18
The Urban Myth Left Unchallenged

• We pass through this world but once. Few
  tragedies can be more extensive than the stunting
  of life, few injustices deeper than the denial of
  an opportunity to strive or even to hope, by a
  limit imposed from without, but falsely
  identified as lying within.

Gould, 2003
19
St. Louis Public SchoolsUsing Formative
Assessment Data to Improve Instruction

• CCSSO Summer Leadership Training Conference
• August 2, 2006
• Joseph A. Jacovino, Ed.D.
• Chief Accountability Officer

20
The Need for a Data-Driven System

• 42 of SLPS schools do not meet the AYP
  requirements of No Child Left Behind
• 5 district middle schools have been placed in
  Corrective Action
• Significant achievement gaps exist among
  disaggregated student groups in middle and high
  schools
• 10 district high schools are among the states
  lowest performers and have not made academic
  progress in over 5 years
• 15 of SLPS schools could face severe
  restructuring sanctions during the 2006-2007
  school year
• There is a lack of district-wide data integrity
  and reliability
• There is not a consistent and uniform use of data
  to drive school improvement planning and district
  reform efforts

21
The Need for a Data-Driven System

• 39 district schools are identified as being in
  Corrective Action and Needs Improvement
  categories of NCLB
• Of these 39 schools, 14 schools are eligible for
  governance changes in 2006-07 and currently face
  severe sanctions
• 2 elementary schools have not met AYP goals for
  three consecutive years and face severe sanctions
  if they do not meet AYP in 2006
• 21 schools have not met AYP goals for two
  consecutive years (12 middle 9 elementary
  schools)
• 57 schools made AYP in 2005, an increase from 47
  in 2004 (including 8 schools on the needs
  improvement list for a first year)

22
School Performance and its Implications for the
St. Louis Public Schools
How will we monitor school performance? Step I
The establishment of a unified data-driven
accountability system for all St. Louis Public
Schools.
How are our schools performing? Step 2 Build a
profile for each schools present academic and
demographic performance.
What needs to be done based on the performance
data? Step 3 The implications of present
performance for reform efforts.
23
Office of Accountability, Assessment,
Intervention

• Coordinates the Districts efforts for
  academic improvement and sustained growth through
  a system of assessing, evaluating, monitoring,
  and providing assistance to all schools
• Implements a uniform accountability system for
  all schools including those that are privately
  managed, restructured, and charter schools
• Implements and monitors the district-wide
  assessment system (including a benchmark
  assessments) for all schools and students and
  reports the results to schools, clusters,
  parents, and the public
• Implements School Performance Teams to target and
  support interventions to improve the academic
  performance of the districts 39 lowest
  performing schools
• Monitors the implementation of NCLB guidelines
  and ensures compliance with federal and state
  mandated programming

24
Uniform Accountability System
What accountability measures are in place for
each school operating in the district?
25
How are our schools performing?
26
Spring 2005 MAP Results

• Used for Adequate Yearly Progress (AYP) as part
  of the federal NCLB legislation
• Are measured in terms of proficiency (targets
  were 26.6 in communication arts, 17.5 in math)
• District proficiency is 18.9 in communication
  arts, 16.9 in math
• 57 (of 88) district schools made AYP (8 schools
  for a first year), compared to 47 in 2004
• Elementary schools have shown significant
  improvement in communication arts and math scores
• In communication arts, elementary schools have
  closed the achievement gap and are above the
  state achievement target

27
Fall 2005 TerraNova Results

• In Fall 2005 SLPS assessed over 24,000 Grade 2-10
  students in reading, mathematics, science
  social studies
• In Fall 2005 in grades 2-10, 72 of students
  district-wide are performing below the national
  average in reading, 76 in math 77 in science
• Use to track the yearly performance of a highly
  mobile student population (33 annually in
  elementary and middle schools 75 annually in
  high schools)
• Use results diagnostically in conjunction with
  district-wide benchmark assessments to align
  learning needs to the monitoring of curriculum
  and instruction

28
Benchmark Assessments

• Developed district-wide on-demand uniform
  assessment system to
• Monitor and support the school-wide
  implementation of the districts new core
  curriculum and instructional materials
• Assess all Grade 3-12 students at six week
  intervals throughout the school year
• Assess all students on the concepts, skills, and
  knowledge embedded in the scope and sequence of
  the curriculum at all grade levels
• Assess all students on the grade level
  expectations embedded in the Missouri Show Me
  Standards
• Review the results of numerous assessments to
  gain a more accurate picture of individual
  student, class, school, and district performance
• Pinpoint specific skills needed for in-depth
  individual/classroom instruction
• Monitor student progress across multiple levels
  of performance
• Provide data for specific and targeted
  professional development for teachers to more
  effectively deliver the curriculum and support
  improved student achievement

29
Utilizing Data forSchool Intervention Support

• School Performance Teams (SPT) are a requirement
  of the SLPS Accountability System for schools in
  Needs Improvement and Corrective Action
  categories of NCLB
• SPTs are a case management approach that use
  formative assessment data for school assistance
  to improve school academic performance through
  the collaboration of central and area office
  staff with school staff

30
School Performance Teams

• The SPT process directly impacts 39
  low-performing district elementary, middle, and
  high schools
• 12 elementary schools
• 21 of district total 6 made AYP for a first
  year in 2005
• 17 middle schools
• 81 of district total 2 made AYP for a first
  year in 2005 4 are qualified for restructuring
• 10 senior high schools
• 91 of district total 10 qualify for
  restructuring

31
School Performance Teams

• School Performance Teams (SPT) consistently use
  formative data
• SPT is an on-going process by which a school
  collects and analyzes data to inform school
  improvement planning with direct support from the
  Central and Cluster Offices
• SPT is based on transparent criteria and
  indicators
• SPT seeks multiple sources of evidence to
  understand how well a school is working and
  students are performing
• SPT ensures that key questions, standards, and
  indicators are used to evaluate the progress of
  all schools
• SPT coordinates supports for implementing school
  improvement plans to foster improved student
  achievement
• SPT collects data to prioritize and mobilize the
  necessary resources to address school needs
  identified through a rigorous school assistance
  process

32
School Performance Team Inquiry

• What do available formative assessment data tell
  us about students strengths and needs?
• What do we need to understand about formative
  assessment to use this data to help students?
• How do we link demographic data to student data
  and why?
• What do we need to know about demographic data to
  help specific student groups?
• How well do our school practices reflect what
  research tells us successful urban schools do?

33

School Performance Team Inquiry

• What do we need to know, change, and do to better
  adopt effective school practices?
• What capacity and resources does our school have
  to implement change?
• What are the most important things to affect
  improved student learning that are also within
  our capacity and resources to influence?
• What are specific goals, benchmarks, and
  strategies to get the most important things done?

34
Implementing NCLB requires a formative
data-driven accountability system

• offering increased school choice options for all
  students
• offering quality supplemental educational
  tutoring services for all eligible students to
  close the achievement gap
• improving teacher quality through the
  implementation of effective recruitment,
  retention, and support strategies
• offering viable alternative education for
  disruptive students
• analyzing the findings from the SPT process using
  a variety of trend reports to leverage needed
  resources for school improvement
• coordinating support from area and central
  offices for identified common improvement areas
• supporting principals and school teams in an
  ongoing process to use data and effective
  monitoring systems for improved instruction and
  academic achievement
• replicating verified models of academic and
  social success

35
Aligning District Education Reforms

• Implementing a uniform standards-based core
  curriculum
• Implementing uniform instructional strategies,
  materials, and textbooks
• Increasing instructional time for reading and
  mathematics
• Providing school-based coaches to support
  curriculum reform
• Increasing professional development for staff
• Implementing a system-wide benchmark assessment
  system
• Creating smaller K-8 schools as part of middle
  grades reform
• Implementing significant secondary education
  reforms
• Implementing a rigorous promotion and graduation
  policy
• Implementing mandatory after school and summer
  school programming for remediation and enrichment

36
What does a data-driven accountability system
achieve?
Increased use of formative data for strategic
decision-making
37
The Use of Benchmarks in the School District of
Philadelphia
Cleopatra Figgures, Ed.D
CCSSO August 2006
38
Why Benchmarks?

• To provide systemic and periodic information
• To identify student strengths and weakness
• To support increased student learning and
  achievement
• Allows for differentiation and individualization

39
03-04 Core Curriculum/ IMS introduced
District-wide Math and Literacy (Grades K-9)
03-04 Benchmark testing/ IMS introduced
District-wide Math and Literacy (Grades 2-9)
04-05 Core Curriculum introduced grades 10-12
Benchmark Pilot In 21 low-performing schools
04-05 Benchmark testing introduced Grades 10-11
40
Compelling Issues

• A need for more frequent assessment of student
  mastery
• Wanted to ensure the SDP standards-aligned core
  curriculum was provided to all students
• Needed a process for critical intervention
• Wanted to be able to tailor instruction to meet
  instructional needs

41
What Are the Benchmarks?

• A component of the Core Curriculum which allows
  teachers to assess whether students are
  proficient on the objectives and standards
  addressed for each six week time period
• Most are done as pen and pencil tests, although
  one Region has the capability to take online
  benchmark tests

42
Benchmark FAQs

• Do the Benchmarks refer to the teaching
  objectives?
• Do they assess only skills that have been
  mastered?
• What do schools do with the Benchmark tests once
  they have been given?
• How are the Benchmarks related to teaching to
  proficiency?

43
SDP Benchmark Design

• Closely aligned to the SDP core curriculum
• Assesses the knowledge and skills taught in a
  subject
• Drives instruction for teachers and helps promote
  standards and content mastery in students
• Serves as a diagnostic tool to measure progress
  and target areas of strength and need for
  improvement
• CI Design the blueprints test vendors submit
  items aligned to the blueprints

44
General Approach

• Based on the SDP core curriculum
• Provides a snapshot of student learning every six
  weeks different content
• Are written to measure student mastery of
  educational objectives listed in the core
  curriculum planning and scheduling timeline
• Not written to have predictive validity for the
  state assessment
• Used during SIP discussions

45
What is Working

• Teachers get immediate feedback from a
  formative assessment
• Benchmarks allow for reflection on instruction
• Ideas on alternative strategies are discussed
  and exchanged
• Benchmarks are viewed as an opportunity for
  collaboration

46
Implementation Process

• One sub-region in the SDP (22 schools) uses
  online Benchmarks
• All students in Grades 3-12 take Benchmarks in
  reading and math
• Pen and Pencil test results are returned within
  three days
• Benchmark results are posted on SchoolNet for
  each period

47
How the Benchmarks are Used

• As part of the Principals Management Process
• Monthly regional meetings with CAO
• Grade group meetings
• Instructional planning
• School Improvement Plans

48
Challenges

• To not use benchmarks to predict performance on
  other assessments
• For administrators to not use results as an
  evaluation tool
• Timeline inconsistencies with core curriculum
• Poor item construction

49
Challenges

• Lack of time for test design and review
• Relationship between test vendors
• Lack of really new items
• SDP and test vendor staff turnover

50

• Suggestions
• Establish benchmarks to measure success of
  instruction
• Review plans for collecting and analysis
  benchmark data
• Establish outcome benchmarks for improvement
  initiatives

51
Suggestions

• Frequently revisit Benchmark report formats
• Use common language with other assessments given
  in the SDP
• Establish implementation benchmarks to measure
  whether schools offer learning experiences as
  intended

52
Things to Do

• To provide on-going professional development on
  the use of benchmark data
• To use benchmark data for improved instruction
  and academic achievement
• Have SDP staff periodically meet to discuss
  benchmark items as writers and reviewers

53
Benchmark Tests
54
Benchmark Testing Online Student Item Analysis
55
Item Analysis
Standard statement
Student score
56
Benchmark Data Analysis

• Benchmark Test Data Analysis
• Cumulative Benchmark Test Results October 2004
  June 2005
• What do the data show about the school average
  percentage of items correct compared to the
  district average?
• For each Reading Standard, circle the grade
  level(s) where the average percentage of items
  correct was low( below 60) and/or substantially
  lower (5-10 point difference) that the average
  for the District
• Grade Level
• 3 4 5 6 7 8 9 10 1.1-Learning to
  Read Independently
• 3 4 5 6 7 8 9 10 1.2-Read
  Critically in All Content Areas
• 3 4 5 6 7 8 9 10 1.3-Reading,
  Analyzing and Interpreting Literature
• 3 4 5 6 7 8 9 10 1.7
  Characteristics and Functions of the English
  Language
• 3 4 5 6 7 8 9 10 1.8 Research
• Comments
• __________________________________________________
  __________________________________________________
  __________________________________________________
  __________________________________________________
  __________

57
Data AnalysisProtocolBenchmark Reflection
58
Question and Answer Session