Who Benefits from AfterSchool Tutoring with Supplemental Education Services SES Funding: A TwoYear S

1
Who Benefits from After-School Tutoring with
Supplemental Education Services (SES) Funding A
Two-Year Study of SES in Georgia

• Southeast Evaluation Association 18th Annual
  Conference February 2-3, 2006 Tallahassee, FL
• Presenters
• Dorothy Harnish, College of Education, UGA
• Joy Johnson, Georgia Department of Education

2
Overview of Presentation

• Supplemental Educational Services
• Overview of SES
• SES and No Child Left Behind
• Need for Evaluation
• Two Year Study of SES
• Purpose
• Data Collection
• Key Findings
• Emerging Issues
• Looking Ahead Evaluation 2005-2006

3
Overview of SES

• Supplemental educational services (SES)
• Free tutoring services that must be offered to
  low-income children who attend a Title I school
  that is in at least its second year of Needs
  Improvement status

4
Overview of SES

• SES must provide instruction that is consistent
  with the instructional program of the Local
  Educational Agency (LEA) and aligned to state
  standards.
• Reading, Language Arts, and/ or Mathematics
• SES must be offered outside of the regular school
  day - before or after school, on weekends, or in
  the summer.

5
No Child Left Behind and SES

• Roles, Responsibilities, and Requirements
• State
• LEA
• Parents
• Providers

6
Role of State

• Identify both private and public entities that
  can offer tutoring and remedial services to
  eligible students.
• Maintain an updated list of approved providers.
  
• Recommend for removal ineffective providers.
• Develop, implement, and publicly report on
  standards and techniques for monitoring the
  quality and effectiveness of the services.
• NCLB 2001, Section 1116 (e)

7
Role of Local Educational Agency (LEA)

• Notify parents of right to choose SES for their
  child.
• Help parents choose a provider that best meets
  the academic needs of their child.
• Enter into an agreement (contract) with the
  provider selected by the parent.
• Monitor the provider, as needed, to ensure
  compliance with the terms of contract.

8
Role of Parent

• 1. Request SES for their child.
• 2. Select provider from the State Board of
  Education (SBOE) approved providers list.
• 3. Transport students to and from the place of
  service when not provided by the provider.
• 4. Work with the provider to set achievement
  goals for the student.
• 5. Maintain open communication with providers
  about student progress.

9
Role of Provider

• Establish fee structure for cost of services
  (i.e. hourly rate, number of sessions, etc.).
• Set specific achievement goals for each student
  in collaboration with each students parent.
• Describe how each students progress will be
  measured and how parents and teachers will be
  regularly informed of the students progress.

10
Role of Provider

• Establish a timeline for improving achievement.
• Ensure that instruction is consistent with
  student achievement goals.
• Agree to abide by the Providers Code of Ethics
• Other requirements

11
Funding

• Unless a lesser amount is needed, LEAs are
  required to spend an amount up to 20 percent of
  their Title I, Part A allocation for SES to
  provide or pay for transportation for students
  opting for public school choice and to satisfy
  requests for SES.
• Eligible students receive an SES allocation (e.g.
  1500.00) that varies by LEA

12
Evaluation Need

• Multiple Sources
• Meaningful
• Triangulated
• Data-driven
• Defensible

13
Purpose of the Study

• Provide the state Title I Office with information
  about how SES is being implemented in Georgia,
  who is being served, and the nature and quality
  of SES provider services
• Develop and use a uniform statewide system to
  collect and report data on SES implementation,
  processes, and issues of concern
• 
  (Contd)

14
Purpose of Study (Contd)

• Provide a statewide perspective from key
  stakeholder groups about SES implementation,
  quality, effectiveness, and satisfaction.
• Recommend to GDOE areas where improvements are
  needed in SES to address quality and
  effectiveness of SES and providers

15
Participants

• School systems and schools required to offer
  SES
• 140 systems, 385 schools in 2003-04
• 100 systems, 203 schools in 2004-05
• Title I Directors at school systems required to
  offer SES
• Parents of students receiving SES
• Teachers of students receiving SES
• Principals at schools required to offer SES
• SES Providers in Georgia

16
Data Collection Methods

• Web-based data collection from all districts
  schools required to offer SES in 2003-04 and
  2004-05
• Onsite visits to sample of school systems
• One system in each of 7 regions, Spring 2004
• Highly successful systems (3-5), Spring 2005
• Surveys of SES stakeholder groups
• Parents of SES students
• Teachers of SES students
• Title I Directors/SES Coordinators
• Spring 2005

17
Data Collection Online Surveys

• On-line Web-based data input via Internet
• Structured and open-ended questions completed by
  district Title I Director
• Three questionnaires System, School, Provider
• SYSTEM LEVEL SURVEY
• Amount of SES funding and its allocation to SES
  providers private, non-profits, school system
• Informing parents of student eligibility
• Prioritizing eligible students
• Monitoring SES providers
• Transportation to SES

18
Data Collection Online Surveys

• SCHOOL LEVEL SURVEY
• Who is being served by SES?
• Number of SES-eligible students
• Number of parents requesting SES
• Number of students receiving SES
• Demographic data for students eligible,
  requesting, and receiving SES
• grade level, ethnic/racial group, gender, limited
  English proficiency, migrant students, students
  with disabilities, Early Intervention Program

19
Data Collection Online Surveys

• PROVIDER LEVEL SURVEY
• Schools and students served by each provider,
  including student demographics
• Number of students receiving Mathematics and
  Reading/Language Arts SES
• Location of SES sessions
• Time of SES sessions
• Format of SES sessions
• Student attendance at SES sessions
• Recommendation on continuation of each provider
  (end-of-year survey)

20
Data Collection Stakeholder Surveys

• Online/web-based and paper questionnaires for key
  stakeholder groups
• Parents of students receiving SES (790
  responses)
• Teachers of students receiving SES (1,529
  responses)
• Title I system-level coordinators (131
  responses)

21
Data Collection Stakeholder Surveys

• Topics addressed by survey questions
• Provider interaction with teachers and parents
• Teacher involvement in developing learning plans
• Providers services, delivery of SES instruction
• Services for special needs students
• Compliance of providers with state and federal
  regulations
• Perceptions of impact of SES on students
• Satisfaction with SES

22
Data Collection On-site Visits

• Purpose
• understand how SES was being implemented
• identify issues, concerns, benefits from
  perspective of all SES stakeholders
• Two-day, on-site visit by team of UGA researchers
  to 7 systems (one per region, diverse
  size/location)

23
Data Collection On Site Visits

• Focus group discussions on-site
• Teachers of SES students
• Parents of SES students
• Administrators of SES schools
• Providers of SES services
• Interviews with system Title I Director/SES
  Coordinator, system staff working with SES

24
Data Collection On-site Visits

• Site visit data collection focused on
• Communications among key stakeholders
• Administration of SES
• Contracting with SES Providers
• Location, timing, transportation for SES sessions
• Participation by students
• Impacts on student learning and achievement
• Quality of SES
• Recommendations for improvements

25
Key FindingsWho is being served by SES?

• Year One (2003-04)
• 10.9 of eligible students received SES
• Grades 3, 4 more likely to participate
• Participation rate of Black students exceeded
  state average Hispanics were at state average
• Participation rate of students with disabilities
  was less than state avg.

• Year Two (2004-05)
• 9 of eligible students received SES
• Grades 6, 7, 8 more likely to participate
• Participation rate of Black and Hispanic students
  exceeded state average
• Participation rate of student with disabilities
  was less than state avg.

26
Key Findings How is SES being implemented?

• Year One (2003-04)
• 4 of Title I funds were spent on SES
• School districts were 24 of all providers but
  served 65 of all SES students
• Districts used multiple means of informing
  parents
• Most did not provide transportation for students

• Year Two (2004-05)
• 2.4 of Title I funds were spent on SES
• School districts were 6 of providers and served
  9 of SES students for-profit providers served
  81 of SES students
• Majority of districts allowed multiple enrollment
  periods, used parent liaisons
• Few districts provided transportation

27
Key Findings How are students served by SES
providers?

• Year One (2003-04)
• Individual and group tutoring most common methods
• 89 of SES is after school, 36 on Saturdays
• School and provider sites used equally for SES
  35 of private providers use schools for SES
• More receive reading/language arts than math SES
• Black SES students a higher portion of those
  served by private providers districts serve more
  White and Hispanic SES students

• Year Two (2004-05)
• Individual and group tutoring most common methods
• 92 of SES is after school, 45 on Saturdays
• Provider sites, community, and schools used
  equally as locations for SES
• More receive reading/language arts than math SES
• Black students a higher portion of SES students
  with non-profit and for-profit providers than
  with school district providers

28
Key Findings How are SES providers and services
assessed?

• Year One (2003-04)
• District monitoring relied on provider reports,
  parent/teacher feedback half used onsite
  observations of SES
• Title I directors assessment of SES providers
  was mostly positive, some problems
• Student attendance was not a problem 69 of
  students attended SES sessions all/most of the
  time

• Year Two (2004-05)
• District monitoring relied on provider reports
  and assessments of student progress half used
  onsite observations of SES 64 used standardized
  student test scores
• Title I directors assessment of SES providers
  was mostly positive (78), some problems
• Student attendance at SES sessions was not a
  problem overall

29
Emerging Issues

• Parent awareness and understanding of SES
  critical for student access to services need for
  proactive role
• Participation by most needy students, barriers
• Redefined roles for private sector and public
  schools in addressing needs of low-achieving
  students
  
• (contd)

30
Emerging Issues (Contd)

• Evaluating the quality and impact of SES
• on student and school performance
• Gathering and interpreting information on a
  large-scale, multi-site program within
  controversial climate of NCLB legislation
• and regulations

31
Evaluation Use by GDOE

• Additional Staff to Monitor Providers
• SES Standards for Monitoring Providers
• Parent Awareness
• SES Media Campaign Get the Word Out!
• Participation Rate Data
• Technical Assistance for LEAs

32

• Georgias Framework for Supplemental Educational
  Services (SES)

Effectiveness
Customer Service
Service Delivery
Has the provider contributed to increasing
student achievement?
Are stakeholders pleased with the program?
Are LEAs and providers implementing and
complying with SES rules and regulations?
Who Statewide by Provider Measure CRCT
GHSGT Data Collection and Analysis UGA
LEA, Provider, Parent(s)/Legal
Guardian(s) Middle/High School Students Data
Collection and Analysis UGA
Annual Monitoring SEA LEA Provider Data
Collection and Analysis GDOE
33
Looking AheadEvaluation 2005-2006

• New Focus
• Student Achievement
• Comparative Study of SES Participants
• vs. SES Eligible Non-participants
• Statewide
• Individual Provider
• Customer Satisfaction
• Survey of Middle/ High School Students
• Service Delivery
• Summary of On-site Monitoring Visits

34
SES EVALUATION SCHEMATA
Meets and lt 2 yr
SD/ CS
Remain on List
Meets
Meets and 2 yr
EFF
Does Not Meet
Does Not Meet
Recommendation for Removal
35
Questions and Answers
36
Contact Information

• Dr. Dorothy Harnish
• Director, Occupational Research Group
• College of Education
• University of Georgia, Athens, GA
• [phone]
• harnish_at_uga.edu
• Joy Johnson
• Education Research and Evaluation Specialist
• Title I Programs
• Georgia Department of Education
• [phone]
• joyjohns_at_doe.k12.ga.us