National Institute for Literacy Community Literacy Initiative Evaluation of Community Literacy Coali

1
National Institute for Literacy Community
Literacy InitiativeEvaluation of Community
Literacy Coalition Data Tracking PilotsMargaret
DoughtyLiteracy PowerlineRaymond HartRS Hart
and Partners
2
Overview

• Community Literacy
• Background
• Statement of the Problem
• Methodology
• Results/Expected Outcomes
• Questions and Answers

3
Significance of theCommunity Literacy Initiative

• Traditional data tracking, reporting, and
  measures of quality are targeted at the program
  level
• Small programs and small samples often make
  impact studies unfeasible
• Community tracking and evaluation offers new
  opportunities for benchmarking, monitoring
  progress, and measuring the community impact of
  programs

4
What is Community Literacy?

• Community Literacy is the practice of
  incorporating literacy into all community
  initiatives to build healthy neighborhoods,
  strong economies and successful families
• Community Literacy creates a discourse around
  shared problem solving to promote the vision of
  100 percent literacy through 100 percent
  community engagement
• Community Literacy allows people and
  organizations to do together what they cannot do
  alone

5
Impact Model
6
Efforts must be coordinated and well designed!
7
Background
8
Current Reality

• All communities provide literacy services but few
  integrate the services within the structure of
  the community
• This results in a fragmented system driven by
  funding silos that are not aligned to each other
  or to broader community needs
• Communities expect literacy providers to take
  sole responsibility for literacy activities
  rather than fostering a spirit of community
  responsibility

9
Engaging the Whole Community
10
Trends

• Communities are moving away from fragmented
  services (over 70 coalitions and many developing
  collaborations)
• Regional literacy planning is becoming more
  common
• Programmatic infusion of literacy at the local
  level is increasing
• Community literacy coalitions are being initiated
  by coordinated local leadership
• Literacy is not the goal it is the tool to
  accomplish the goal

11
The Strategy of Infusion

• This is the practice of ensuring literacy
  outcomes in all community social services and
  initiatives, for instance
• Including literacy in city and county planning
• Providing family literacy in all programs that
  support family success
• Offering computer, health, workforce or financial
  literacy in non-traditional neighborhood settings
  
• Providing opportunities for 100 engagement

12
Infusion Paradigm
13
Lifelong Learning
14
Mechanism of Transformation
15
Community Collaboration
16
Regional Literacy Planning

• Goal
• To determine the need, identify solutions, and
  implement a collaborative literacy plan with
  community stakeholders, legislators, funders,
  agencies and providers who will create an
  effective, seamless pathway for lifelong learning.

17
(No Transcript)
18
Focus

• High level leadership engagement
• Issue elevation
• Coordinated marketing and messaging
• Aggressive, collaborative resource development
• Lifelong learning to create pipelines for success

• Increased quality of services
• Increased scope and capacity of systems to
  deliver
• Collaborative information/referral (211)
• Centralized tracking and evaluation
• Community will to succeed

19
Statement of the Problem

• Communities are bringing together a service
  delivery network that
• Is fragmented
• Has little tradition of collaboration
• Relies on silo funding
• Has limited community-wide accountability
• Has difficulty showing return on investment

20
The Challenge Delivering thousands of at-risk
children, youth, and adults to labor market
success
??
??
Charter
??
Early Childhood
??
Children Youth
??
??
??
Literacy
GED
??
??
Child welfare
Good jobs
??
??
??
??
Secondary and Post sec credentials

??
??
Contextual learning
Good wages
??
??
??
??
Career exposure
??
??
??
Leadership opportunities
??
??
??
Work experience
Personal Development
Good benefits
??
??
??
??
??
??
Adult Learners
??
Advancement opportunities
K-12 Education
Justice System
21
What is Needed to Demonstrate Progress?

• Common assessments
• Common measures
• Shared tools and tracking resources
• Centralized tracking
• Regular and coordinated evaluation
• Regular feedback to programs, funders and
  communities

22
Spring 06 to Fall 06 Age vs Age Equivalent
Difference for Returning Pre-School Students
23
Comparison of Students in After-school on a
Performance Series Assessment
Figure 1
Figure 2
This trend was seen when analyzing all math
students participating in the study (Figure 1)
and regular math students only (Figure 2).
24
Long Term Projections Summary TableBased on
Scale Score Growth per 30 Hours
25
Measuring Community Outcomes

• In an effort to provide the best support for
  communities to determine both short and long term
  impact the following steps are suggested
• Evaluate data tracking pilots (6 coalitions)
• Develop implementation plan for tracking
• Understanding relationship between tracking
  progress across communities and HIPPA
• Understanding commitment to data management
• Understanding new role of reporting community
  outcomes

26
Measuring Community Outcomes

• Evaluation Should Measure
• Effectiveness of instruction across the coalition
• Recruitment, training and placement of volunteers
  and mentors
• Tracking of learner progress through pipelines
  (both recruitment and retention)
• Transition to work and continuing education
• Resource allocation and return on investment data

27
Measuring Community Outcomes

• To evaluate the first round of community literacy
  tracking pilots implementing the LACES system,
  evaluators should collect data on
• Ease of access and training
• Development of policy and procedures for sharing
  data, including securing confidential information
  and addressing concerns about data ownership and
  access.
• Accountability and program effectiveness
• Analysis of learner gain data
• Recruitment and retention data
• Best practices and cost effectiveness
• Data on instructor and volunteer professional
  development

28
Measuring Community Outcomes

• The evaluation questions of interest are
• What are the common elements of individuals
  (administrators, instructors and staff)
  experiences as a result of the LACES
  implementation?
• To what extent are data driven decisions enhanced
  and measures of program outcomes improved as a
  result of LACES implementation?
• What are the changes in instructional and program
  practices as a result of the LACES
  implementation?
• To what extent are programs able to create
  linkages between early childhood, K-12, and Adult
  Education data systems?
• To what extent are programs able to share data
  resources and historical student performance data
  to better serve clients?
• The evaluation should be divided into two phases
  
• conduct a one year study of the six pilot
  implementation projects, and
• conduct a longitudinal study of program use and
  implementation over three years

29
Expected Outcomes

• Evaluation of the LACES system capacity to meet
  community literacy needs
• Development of Quality Indicators for Community
  Literacy Systems
• Guidelines for tracking program and learner
  outcomes across communities
• Presentation of best practices for community
  literacy implementation

30
Proposed Next Steps

• Identify Community Literacy as a high priority
  that addresses NIFLs goals
• Create a Community Literacy Task Force
• Provide support for a Community Literacy research
  agenda
• Engage in an evaluation pilot to support a group
  of coalitions implementing a central tracking
  system

31
Questions and Answers

• Thank You