Building an Effective Evaluation of Your Flexible Learning Program (FLP) Presenter: Sue Hawks Foster, FLP Education Program Specialist Georgia Department of Education sfoster@doe.k12.ga.us (404) 656-2636 - PowerPoint PPT Presentation

About This Presentation
Title:

Building an Effective Evaluation of Your Flexible Learning Program (FLP) Presenter: Sue Hawks Foster, FLP Education Program Specialist Georgia Department of Education sfoster@doe.k12.ga.us (404) 656-2636

Description:

Building an Effective Evaluation of Your Flexible Learning Program (FLP) Presenter: Sue Hawks Foster, FLP Education Program Specialist Georgia Department of Education – PowerPoint PPT presentation

Number of Views:107
Avg rating:3.0/5.0
Slides: 48
Provided by: GaD106
Learn more at: http://www.gcel.org
Category:

less

Transcript and Presenter's Notes

Title: Building an Effective Evaluation of Your Flexible Learning Program (FLP) Presenter: Sue Hawks Foster, FLP Education Program Specialist Georgia Department of Education sfoster@doe.k12.ga.us (404) 656-2636


1
Building an Effective Evaluation of Your Flexible
Learning Program (FLP) PresenterSue Hawks
Foster, FLP Education Program SpecialistGeorgia
Department of Educationsfoster_at_doe.k12.ga.us(404
) 656-2636
2
Agenda FLP Evaluation
  • Federal Regulation Requires Evaluation of the
    Title I Programs
  • FLP Evaluation Requirements-
  • Federal, State, and District
  • Building an Effective Program Evaluation
  • LEA Plan for District FLP Evaluation
  • Performance Based Observation of FLP
  • Most Common Findings FY13
  • Questions and Answers
  • References

3
  • Federal requirements for evaluating title I
    programs

4
Federal Regulation for Title I Programs
  • Federal regulations requires that a school
    operating a Title I program annually evaluate the
    implementation of, and results achieved by, the
    Title I program
  • ESEA Title I program compliance requires written
    program evaluation procedure and program
    evaluation summary

5
Federal Requirements for Evaluation
  • The Title I program evaluation is an annual
    review of Title I activities to determine if they
    are contributing to the desired outcomes
  • Improved student achievement
  • Effective family engagement
  • High quality professional development

6
Federal Requirements for Evaluation
  • Evaluation Components
  • Measures efficacy and impact of the Title I FLP
    program
  • Documents impact of Title I FLP services on
    student learning
  • Informs school and district planning (cycle of
    continuous improvement)
  • Is an advocacy tool
  • Is conducted at end of program year and used for
    the next years planning

7
Federal Requirements for Evaluation
  • Description of
  • Data used
  • Constituents consulted
  • Process used to arrive at evaluation findings
  • Previous data
  • Surveys
  • How findings are utilized for planning and
    improvement

8
Federal Requirements for Evaluation
  • Summary of
  • Program strengths
  • Program weaknesses
  • Subsequent program changes
  • Reallocation of funds

9
Needs Assessment Program Evaluation
  • What are the differences and how do they relate
    to each other?
  • Program evaluation
  • Evaluates Title I program-FLP specifically
  • Completed after the fact (end of year)
  • Helps to plan Title I program for next year
  • including FLP

10
Needs Assessment Program Evaluation
  • Needs assessment
  • Looks at all school data, including Title I
  • Schoolwide data in all subject areas
  • Targeted Assistance data in all subject areas
  • FLP data
  • Completed before beginning of school year
  • Assesses priority needs in district/schools and
    determines where Title I FLP fits in, as well as,
    other resources

11
  • FLP evaluation requirements

12
FLP Evaluation Requirements
  • State Evaluation
  • Participant Feedback
  • Evaluation Question What is the overall
    experience of participants with program?
  • Data Source Student and Parent Surveys
  • Service Delivery
  • Evaluation Question Is the LEAs and programs in
    compliance with laws and regulations?
  • Data Source Annual monitoring data, Program
    documentation, Federal reporting, Public
    reporting, Technical Assistance, etc.

13
State Evaluation contd
  • State Evaluation
  • Effectiveness
  • Evaluation Question Are programs contributing to
    increased student academic achievement and
    performance on state education standards?
  • Data Source Student performance on state tests,
    Pre-post assessment measures of state standards
    and academic skills targeting by programs,
    Performance Flag data, and student growth in
    schools offering FLP
  • Evaluation results would be shared with
    stakeholders and the public and used to inform
    ongoing program improvement

14
State Evaluation contd
  • LEA Evaluation
  • LEAs will be required to evaluate outcomes of FLP
    interventions implemented at schools within their
    district that have been identified
  • LEAs will submit their evaluation plan as part of
    their overall FLP plan for approval by GaDOE
    Title I Specialists

15
State Evaluation contd
  • LEA Evaluation
  • The LEA must submit the following components
    of an
  • evaluation plan with their FLP plan in the
    CLIP for each
  • Priority, Focus, or Alert school within
    their district
  • required to implement FLP
  • Effectiveness target or overall quantifiable goal
    of the program
  • Assessment instrument(s) that will be used to
    measure this target/goal
  • Plan for collecting and maintaining data
  • Data analysis plan

16
State Evaluation contd
  • What do the LEA evaluation requirements mean?
  • Focused, purposeful intervention
  • Tailor evaluation needs of program to match
    program goals and student needs
  • Expand on broadband state assessments to quantify
    benefits of FLP participation
  • Collect meaningful information for decision
    making and continuous program improvement

17
  • Building an effective program evaluation plan

18
Building Effective Program Evaluation
  • Determining Purpose of Evaluation
  • Why should we evaluate our program?
  • What is our program trying to do?
  • What information do our funders expect?
  • What questions do we want to answer?

19
Building Effective Program Evaluation
  • Performance Measures
  • Measures of effort
  • Measures of effect
  • Strong performance measures should
  • Have strong ties to program goals, inputs, and
    outputs
  • Be compatible with age and stage of your program
  • Consider if the data you need are
    available/accessible
  • Yield useful information to the program

20
Building Effective Program Evaluation
  • Assessing Capacity for Evaluation
  • Who should be involved in the process?
  • What resources must be in place?
  • Who will conduct our evaluation?

21
Building Effective Program Evaluation
  • Selecting Evaluation Design
  • What type of evaluation should we conduct?
  • Formative to understand the changes that have
    occurred
  • Summative demonstrate they have achieved the
    standard required
  • What type of data should we collect?
  • Qualitative
  • Quantitative

22
Building Effective Program Evaluation
  • Selecting Evaluation Design (cont.)
  • What is our time frame for collecting data?
  • Causal
  • Experimental
  • Quasi-experimental
  • Non-experimental
  • Descriptive
  • Pre-experimental

23
Building Effective Program Evaluation
  • Choosing evaluation instruments
  • Construct
  • Audience
  • Format
  • Reliability
  • Validity

24
Building Effective Program Evaluation
  • Analyzing data
  • How will data be analyzed and shared?
  • Quantify central evaluation question (and
    supporting questions)
  • Communicating results
  • How?
  • Where?
  • Who?

25
LEA Plan for District flexible learning program
(FLP) evaluation
26
LEA Plan for District FLP Evaluation
  • List the effectiveness target(s) or overall
    quantifiable goal(s) of your programs
  • What are the measureable outcomes that your
    intervention is designed to improve? Must include
    at least one academic achievement indicator

27
LEA Plan for District FLP Evaluation
  • List the assessment instrument(s) that will be
    used to measure each listed effectiveness
    target/goal
  • Include a plan for administering assessment
    instruments and how you will collect and maintain
    data

28
LEA Plan for District FLP Evaluation
  • When will you administer pre- and post-
    assessments?
  • How will you maintain student assessment
    information to ensure confidentiality?
  • How will you determine program effectiveness
    based on program goals and measurement
    instruments listed above?

29
LEA Plan for District FLP Evaluation
  • FLP Program Evaluation Includes Academic Data
    and Narrative
  • FLP Program Summary Chart
  • Parent, Staff and Student Summary Results

30
True Test for FLP
  • Are graduation rates improving?
  • Are CRCT scores increasing?
  • Are EOCT pass rates increasing?
  • Is our CCRPI increasing?

31
Performance based observation (PBO)
32
Performance Based Observation
  • FY13 PBO in Review
  • 12 Districts were selected for PBO as a pilot
  • 16 schools were observed
  • FY14 Review
  • 26 Districts have been selected for review
  • 26 School will be observed

33
Performance Based Observation
  • Components of Performance Based Observation(PBO)
  • Review of Priority, Focus, and Alert School List
  • Select number of districts to be reviewed based
    on the ESEA Flexible Waiver
  • Selection of schools in district to be monitored

34
Performance Based Observation
  • Components of PBO contd
  • Notifications Sent to LEAs
  • Call to Title I Director
  • Follow up Letter to LEA Superintendent, Title I
    Director and Principal of selected schools with
    brief outline of expectations

35
Performance Based Observation
  • Components of PBO contd
  • Agenda Developed by Title I Office
  • Follow up call to LEA by PBO Team Lead before
    visit
  • Addresses and list of staff reviewed
  • Selection of Teachers/Tutors for Interview

36
Performance Based Observation
  • Actual Visit to LEA
  • Interview with LEA Staff
  • Superintendent
  • Title I Director
  • School Principal
  • School FLP Coordinator
  • School FLP Instructors/Tutors (at least 2)
  • Other vital staff as needed

37
Performance Based Observation
  • Actual Visit to LEA
  • Classroom Observation -20 minutes per classroom
  • Talk with students and staff in classroom
  • Observe student and FLP staff interactions
  • Depart Building- Inform building staff we are
    leaving the building

38
Performance Based Observation
  • Follow-up Letter
  • Review of the PBO findings for the district
  • Non-Compliance
  • If a LEA is found out of compliance the district
    will be notified immediately by Program Manger
  • If corrective action is required it will be
    provided in writing and followed up with the FLP
    Specialist and/or the Program Director for FLP

39
  • FY13 FLP Most Common Findings

40
FLPs Most Common Findings
  • Models used for FLP
  • After school programs were used by Middle and
    High Schools
  • Elementary and Middle schools extended the day or
    rotated the scheduled day
  • High schools used Saturday sessions
  • Summer Sessions were held when extra funds were
    available to extend the learning year

41
FLPs Most Common Findings contd
  • Technology Usage Increased
  • Increase Professional Learning
  • Increased Student Instruction Time
  • More contact with parents
  • Reported Increase in Test Scores

42
FLPs Most Common Findings contd
  • Frustration of operating the program by some
    districts because it was so new
  • Large amounts of carryover because of FLP funds
    were not used last year
  • Special Education Departments not connecting to
    FLP in various districts
  • Lack of staff in some districts to serve students

43
FLPs Most Common Findings in Plans
  • Lack of understanding of FLP purpose
  • Did not provide performance goals for plans
  • Determining the needs assessment was not fully
    understood to determine subject area
  • Many did not identify all subgroups within the
    schools
  • Schools did not understand providing data for
    more than one year in order to provide a trend

44
FLPs Most Common Findings in Plans
  • Special Education Departments were not really
    aware of what was going on with Students with
    Disabilities (SWD) in FLP.
  • Student ratio needs to be smaller
  • Individual Learning Plans needed to be developed
    and followed up
  • Documentation of Multiple Selection Criteria was
    lacking in some districts
  • Lack of understanding Professional Learning

45
Questions and Answers
46
References
  • Afterschool Evaluation 101 How to Evaluate an
    Expanded Learning Program, Erin Harris, Harvard
    Family Research Project, 2011 http//www.hfrp.org/
  • Doing What Works Clearinghouse,
    USED -http//ies.ed.gov/ncee/wwc/
  • Preschool for California's Children
  • Hfrp_pubs_at_gse.harvard.edu
  • Harvard Family Research Project, 2013
  • http//www.hfrp.org/evaluation/overview
  • Georgia Department of Education, FLP Report
  • http//www.gadoe.org/School-Improvement/Federal-Pr
    ograms/Pages/default.aspx

47
Contact Information
  • Sue Hawks Foster
  • FLP Education Specialist
  • sfoster_at_doe.k12.ga.us
  • 404-656-2636
Write a Comment
User Comments (0)
About PowerShow.com