Scaling Up and Evaluating Professional Development: How the Georgia SIG Has Impacted Hundreds of Sch

1
Scaling Up and Evaluating Professional
Development How the Georgia SIG Has Impacted
Hundreds of Schools through Professional
Development Initiatives
2

• Georgia Department of Education
• Marlene Bryar, Director Division for
  Exceptional Students
• John OConnor, Education Program Manager
• Ginny OConnell, Education Program Specialis
• North Georgia Learning Resources System
• Laura Brown, Director
• White County Middle School
• Holly Mauney, Assistant Principal
• University of Georgia
• Bill Swan, SIG External Evaluator

3

• Marlene Bryar,
• Director, Division for Exceptional Students
• Georgia Department of Education

4
Our Story between 1999 - 2005

• We have moved from
• One shot workshops to..long term, job embedded
  professional development.
• Satisfaction surveys to.evaluating the impact
  on adult practices and student performances.
• Small, professional development initiatives
  to..large scale initiatives.

5
We are also going to tell you about

• Struggles and barriers.
• Future plans to coordinate with the GDOE
  Technology Division to conduct large scale,
  cohort analysis evaluations.
• How we have taken one project and how it has
  scaled up over the last several years.

6
Background Information

• In 1999 first Georgia SIG.
• In 2004 second Georgia SIG.
• Georgia DOE also supports the Georgia Learning
  Resources System (GLRS)

7
Georgia Learning Resources System (GLRS)

• 17 centers across the state
• Annual budget of approximately 5 million
• Approximately 30 professional staff members

8
Due to the Georgia SIG in 1999, we had a MAJOR
shift in philosophy

• We could no longer evaluate the effectiveness our
  professional development initiatives solely by
• Counting the number of participants
• Passing out surveys that asked the participants
• How did you like the content of the workshop?
• How did you like the presenter?
• How did you like the facilities of the workshop?

9
We had to evaluate professional development by
determining

• If adults changed their practices.
• If students exhibited improved performance.

10
We received BACKLASH from professional
development providers. They said,

• We can never claim causality between
  professional development efforts and adult and
  student outcomes.
• As professional development providers, we have
  no control over what actually happens in
  classrooms.
• We dont have any power over actual
  implementation.

11
Our responses

• We are not conducting research, but evaluation.
• We want evidence that we are contributing to
  positive change, not proof.
• We need summative and formative assessments.
• We need to work on contextual factors in schools,
  not just content.

12
John OConnor, State Improvement Grant
Director Education Program Manager for
Professional Development, including the GLRS
network Georgia Department of Education
13
1999 Our First Efforts at Changing Professional
Development Evaluations

• One-shot workshops continued, but changed
  evaluations.
• Sent follow-up workshops 6 weeks after a
  workshop.
• NOT VERY EFFECTIVE!!!!

14
We realized we had to change our perception of
what constitutes effective professional
development. That would include changing the
evaluation process.
15
Quick Activity What constitutes effective
professional development?

• Split the room in half A B

16
A Group

• Spend about 3 minutes solving this challenge.
• A national chain of martial arts studios wants
  you to be their spokesperson.
• You dont have any background in Martial Arts.
• In two years, they want you to earn a black belt.
• Then, you will be able to show others, through
  commercials, how an average person can become a
  martial arts expert.
• If you meet this challenge, you will have a very
  lucrative contract.
• What steps would you take over the next two years
  to meet this challenge?

17
B Group

• 3 minutes
• An international company that promotes
  bilingualism also needs a spokesperson.
• You only speak English and have no experience
  with other languages.
• In two years, they want you to become extremely
  proficient in Spanish.
• Then, you will be able to show others, through
  commercials, how bilingualism is achievable.
• If you meet this challenge, you will receive an
  all expense paid, 6-week trip to several Spanish
  speaking countries.
• What steps would you take over the next two years
  to meet this challenge?

18
What did you come up with?
19
Gets lots of buy-in and determine necessary
resources
Set clear goals and determine summative
evaluations
Determine validated interventions
Conduct needs assessment
Determine short-term objectives and formative
evaluations
Determine practice schedule professional
learning community
20

• Requirement Every professional development
  initiative funded by the Georgia Division for
  Exceptional Students must include each of those
  elements.

21
The Story of One Project

• In 2000, Georgia had very weak LRE data.
• 1999 Georgia SIG Goal Increase the percentage
  of time students with disabilities are educated
  in general education settings.

22
In 2001, partnered with 6 schools to meet the LRE
goal.

• We
• Conducted needs assessment by analyzing the LRE
  data from each participating school.
• Partnered with the LEA and school administrators.
• Provided ongoing training on administrative
  issues (e.g., school scheduling, impact on school
  finances).
• Provided ongoing training on co-teaching, IEP
  planning, and differentiated instruction.
• Used LRE data both formatively and summatively to
  check progress.

23
Outcomes

• The data from those schools was positive.
• More students with disabilities were being
  educated in general education settings.

24
Making Progress

• Over the next few years, we became more efficient
  at scaling up and evaluating the impact of the
  project.

25
Laura Brown, Director North Georgia GLRS (one of
the 17 GLRS sites)
26
North GLRS Approach to Regional LRE Training
27

• Partnership with Dr. Melissa Hughes and Dr. Paul
  Alberto
• Georgia Center for Collaborative Education, SIG
  funded TA center

28
Challenges to Overcome

• Obtaining Superintendents support and buy-in
• Rolling out quality regional training in a
  relatively short period of time
• Measuring results

29
Superintendents Support

• LRE selected as a targeted goal for entire state
  by state Director for Exceptional Students
• Local Superintendents involved
• Capitalizing on existing pilots
• Meeting with all superintendents as a group
• Sharing information about upcoming plans for
  special education within the state
• Requesting written commitment letters specifying
  all stakeholders interests and commitments
• 100 of Superintendents committed their support

30
Regional Training Plan

• Central Office and School-Based administrators
• LRE Facilitators
• Redelivery for Teachers
• Year 2 Training

31
Administrative Training

• Central Office and School-Based administrators
• 3 Phase Training Schedule
• 1. Decision Making Model for IEP
• GA funding model and student reporting
• Introduction to Co-Teaching
• 2. Building master schedules for schools
• 3. Individualized scheduling for selected
  sites

32
LRE Facilitator Training

• Each system and/or school created LRE
  Facilitation Teams
• Teams consisted of an administrator, general
  education, and special education representatives
• Teams responsible for redelivery to school sites

33
North GLRS

• Resource Tools for Facilitators
• Power Point Presentation on CD
• Black Line Masters for Transparencies
• Manual for Facilitators (presentation guidelines,
  data collection instructions, scripted narrative
  for presentation)
• Handouts for participants
• Nonflexible dates for completion of redelivery
• Intensive 1 day training

34
Data from Training

• 354 administrators
• 6386 school staff (certified staff required)
• 129 school sites
• 14 school systems
• Training timeline
• October through December, 2003

35
Project Goal and Outcomes

• Goal
• Increase the percentage of time SWD are educated
  in general education settings as measured by the
  environmental data submitted by systems on the
  December 1 Report.

36

• Outcomes
• 93 of the 14 systems making up the North GLRS
  region demonstrated an increase in the time SWD
  received special education services from the FY
  04 school year to the FY 05 school year
• Academic achievement performance of SWD was
  compared for grades 1-8 on scores on the GA CRCT
  in Reading, English/Language Arts, and
  Mathematics for 2002, 2003, and 2004.
  Statistically significant improvement was noted
  in scores of SWD in all grade levels and content
  except 7th grade reading.

37
(No Transcript)
38
of Schools Who Made AYP - 2004
39
of SWD Who Did Not Make AYP From 2004 to 2005
(Decrease desired)
40

• How did this project impact a particular school?

41
Holly Mauney, Assistant Principal White County
Middle School
42

• On-site Coaching
• Initial conversation with school administration
• Developed goals
• to utilize special education teachers in
  co-teaching settings most effectively
• to increase co-teaching teams knowledge of
  co-teaching strategies
• to implement a variety of co-teaching strategies
  in the classroom

43
Professional Learning

• Action Plan and Time Line Phase One
• Presentation to co-teaching teams by grade level
• Sharing video clips of teachers modeling various
  co-teaching strategies
• Homework implement one co-teaching strategy and
  be prepared to discuss celebrations and concerns
  during visit two

44
Professional Learning

• Action Plan Phases Two Three
• Observation of co-teaching teams
• Debriefing with each team to discuss strategy
  implemented and concerns

45
Outcomes

• 52 of SWD educated in general education settings
  met or exceeded standards on CRCT.
• 35 of SWD educated in pull out classes met or
  exceeded standards on CRCT.

46
Capacity Building

• Project and philosophy is embedded in the school
  district.

47
Celebrations

• Special education teachers utilized most
  effectively
• Increase of students with disabilities (SWD)
  were educated in general ed classes and were
  exposed to a more rigorous curriculum.
• Increased student achievement.

48
2005-2006Action Plan

• Establish planning time for co-teaching teams
  each grading period
• Examine and evaluate accommodations currently in
  place, making adjustments as needed
• Evaluate grading practices for special education
  students

49

• The LRE project is continuing to grow in scale.
  It is now referred to as
• Student Achievement in the Least Restrictive
  Environment

50

• Ginny OConnell
• GDOE Co-Director of the SIG/Student Achievement
  in the Least Restrictive Environment initiative

51

• In addition to the 247 schools that participated
  in the project during the first SIG,
  approximately 150 schools have joined the project
  since June 04 (2nd SIG).

52
DOE Co-Directors
53
Challenges

• Maintaining the quality of training with
  train-the-trainer model
• Adapting the training and supports to meet the
  variety of skill levels of participants between
  and within districts and schools
• Refining formative evaluations in order to guide
  mid-course adjustments

54

• We are also becoming more sophisticated as we
  conduct evaluation activities.

55
Bill Swan University of Georgia External State
Improvement Grant Evaluator
56
PROGRAM EVALUATION
57
Transition in Emphasis

• Satisfaction ? Student Change
• (Perception) (Evidence)
• Spray Pray ? Long Term Coaching

58
A Better Way to Gain Evidence

• Analysis of
  Evidence
• Model Student of
• Fidelity Achievement Improvement

59
MODEL FIDELITY

• Is the model (instructional approach or
  innovation) being implemented as it was intended
  to be implemented?

60
Model Fidelity Examples

• LRE Projects On-site visits by coaches
• Ruby Paynes Instructional Framework
• -Observation Scale
• -Artifacts/Conference Scale
• (aha!process.com)

61
Analysis of Student Achievement Data

• Cohort Analysis (Same Students
• Over Time) Examples
• 2003-2004 2004-2005 2005-2006 2006-2007
• COHORT I 1st Grade? 2nd Grade? 3rd Grade?
  4th Grade
• COHORT II 2nd Grade? 3rd Grade? 4th Grade?
  5th Grade
• COHORT III 3rd Grade? 4th Grade? 5th Grade?
  6th Grade

62

• B. Design and Statistical Analysis
• Post Test Only Design with Two Groups
• -- Experimental (Implementation of
  model/instructional
• innovation)
• -- Comparison (Implementation of traditional
• instructional approach
• Analysis of Covariance
• -- Dependent Variable Student Achievement
• -- Covariate Prior Student Achievement

63
Analyses of Student Achievement Data--Examples

• LRE ProjectIn process with the Georgia DOE
  Technology Division
• Ruby Paynes Instructional Framework

64

• Questions?