Getting Student Teachers Linkages Right

1
Getting Student Teachers Linkages Right!

• Jeff Watson
• Value-Added Research Center (VARC)
• UW Madison
• Deborah Lindsey
• Milwaukee Public Schools

2
Goals

• Describe why S-T linkages are important
• Define S-T linkages
• Discuss factors that impact S-T linkages
• Present a framework for the types of factors
• Provide examples of analysis methods
• Provide solutions to problems

3
Why care about S-T linkages?

• Several types of data use require knowing
  something about what teachers are teaching what
  kids
• Classroom centered reporting class rosters of
  incoming students
• Fidelity of implementation knowing how
  improvement efforts enacted (formative and
  summative)
• Identifying effective practices grouping
  students into treatment and control groups (not
  teacher evaluation)
• Rigorous program evaluation isolating the
  effects of programming causal attribution

4
Why care about S-T linkages? (cont)

• Classroom value-added analysis attributing
  growth in student achievement to individual
  classrooms
• Cost Accounting of HR expenses below school
  level examining payroll expenditures by
  student/classroom/school rather than
  district-wide
• Targeted resource allocation directing resources
  (e.g., professional development) to specific
  teachers and staff
• Performance based compensation gain or growth
  metrics for groups of kids (not in MPS)

5
Defining Student Teacher Linkages
Student
Teacher
A record of which teachers and staff taught which
students during a school year.
6
Defining Student Teacher Linkages
Student
Teacher
7
Defining Student Teacher Linkages
School
Student
Teacher
8
Defining Student Teacher Linkages
School
Student
Teacher
Course
9
Defining Student Teacher Linkages
School
Student
Teacher
Course
Outcomes Formative assessment, Achievement,
Graduation, Behavior, Post-graduation success
Practice Pedagogy, Curriculum, Tutoring,
Coaching, PD, Scheduling, Class size
10
Four Easy Questions

• What is a student?
• What is a teacher?
• What is a school?
• What is a course?

11
Warning Reality Approaching
School
Student
Teacher
Course
12
movement between schools
School
Teacher
Student
School
13
and movement within schools
School
Student
Teacher
Course
Course
Course
14
and complicated workflows
School enrollment is estimated
Schedule may change based on completion
School
Student
Staffing needs are estimated
Determined sometime between Feb. and July
Course
Maybe hired in Sept.
Teacher
15
(No Transcript)
16
and absence rates
School
Student
Teacher
Extended leave, moved to cover, absences?
Attendance rate, discipline infractions
Course
17
Instructional strategies like grouping,
pull-outs, room aides, team teaching (SAGE)
School
Student
Teacher
Instructional Support staff
Course
Specialists
18
non-traditional schools
School
Student
Non-district employed Teachers
Course
Unique instructional approaches
19
data errors, integration issues, SIS work
arounds
SIS
School
Student
Teacher
Student
Teacher
Matching Records
HR
Course
Creative scheduling
Data entry problems
20
Review of Student Teacher Linkages
Course Scheduling
SIS
Attendance rate, discipline infractions
School
Student
Teacher
Student
Teacher
Extended leave, moved to cover, absences?
Student Enrollment
Matching Records
HR
Course
Instructional support staff
Mobility School
Creative scheduling
Data entry problems
HIRING workflow
Specialists
Unique instructional approaches
School staffing adjustments
21
Analytic Framework
Technology and Tools
Organization
Person
Tasks
Environment
(Balance Theory Smith and Carayon, 1989 Carayon
and Smith, 2000)
22
Transition Slide

• Early work in student teacher links was driven
  by the need / desire to explore classroom
  value-added analysis
• At the time MPS did not record student teacher
  linkages we used a proxy found in a proficiency
  system built on the SIS.

23
Linking S-T

• Focus on grades 3, 4, 5 for 1 year
• 17, 961 students 2113 unmatched with a teacher
  (11 )
• TEST SCORE lt-gt SIS TEACHER DATA lt-gt HR TEACHER
  DATA
• 1788 students unconnected to teacher data (SIS
  bolt on app)
• 325 additional students with SIS teachers data,
  but with no HR teacher data
• Problems
• Multiple ID systems for Teachers
• Mobility
• Inconsistent Demographic data between systems
• 5 schools had disproportionate share of errors
• Late proficiency submissions by some schools

24
(No Transcript)
25
Classroom Sniff Test

• Infer Classrooms
• Build frequency distributions for
• students per classroom
• of classrooms per student per teacher
• of schools per student per teacher
• Apply reasonable thresholds
• E.g., classrooms with less than 5, more than 35
• Group by school and vet with district and school
  staff

26
(No Transcript)
27
Distribution of teachers by of Schools and
of Grades Taught
28
Looking Back Moving Foward

• Benefits of proficiency system as a S-T source
  nearly exhausted
• Degree to which Proficiency teacher equals the
  classroom teacher is unknown
• Need to develop future solutions to capture and
  store S-C-T linkages more systematically

29
Solutions implemented in MPS

• Improving data quality of employee IDs in SIS
• Course lockdown
• Data warehouse upgrade to track course
  assignments queries to extract course
  assignments

30
Improving Teacher IDs in SIS

• Many teachers in SIS could not be linked to HR
  b/c of bad employee number data in SIS
• Focused on workflow of entry point of Teacher
  data in SIS
• Validation of ID form (string length)
• Lookup tables for ID and last name
• Focus on PD for data entry staff and
  documentation through error msgs
• Still allow for work around (000000) for teachers
  yet to be hired or are not in HR database

31
Percent of Teachers with accurate HR data in SIS
32
Improving Data Quality of Course Numbers

• Course numbers not meaningful, courses highly
  varied and not consistent between schools
• Cross departmental participation
• Focus on consolidating course numbers, connecting
  courses to standards, and prescribing general
  syllabi
• Identify key technical and organizational staff
  members
• Targeted high school courses first, followed by
  middle school
• Current catalog is locked, new scheduling is
  being done with new standardized catalog

33
Data Warehouse Development

• Build a consortium with MPS, Versifit
  Technologies (DW vendor), and VARC
• Build trust
• Identify common areas of interest and benefits
• Develop S-T design requirements driven by MPS and
  VARC uses
• Leverage strengths / avoid weaknesses
• Continued communication and feedback

34
Data for Assessing S-T Links

• Enrollment Data
• sIDs, course IDs, course names, course outcome,
  teacher IDs, period, period duration
• Entity Data
• Student, teacher, school
• Transaction Data
• Withdraw, Admittance, Enrollment
• Attendance
• Student
• Teacher

35
Review Solution Strategies

• Have broad participation
• Pick high leverage areas
• Focus on both technical and social aspects
• Implement changes as far upstream as possible
• Make problems visible

36
Questions

• Jeffery Watson
• jgwatson_at_wisc.edu
• 608.263.0436
• Deborah Lindsey
• lindsedl_at_milwaukee.k12.wi.us
• 414.475.8751

37
(No Transcript)