Toward a National Framework for Evidence of Effectiveness of Teacher Education Programs

1
Toward a National Framework for Evidence of
Effectiveness of Teacher Education Programs
May 16, 2008
Mona Wineburg Executive Director CREATE
2
Overview   University-based teacher preparation
programs are under unprecedented scrutiny and
challenge. The entire concept of
university-based teacher preparation is being
challenged.   Programs must provide credible
and persuasive evidence of effectiveness or
risk losing out to a host of existing and
emerging competitors providing alternative
routes to teacher licensure.
3
Producing sound evidence is no trivial matter

• Substantial intervening variables
• Socio-economic status
• School conditions
• Questions about appropriate measures of pupil
  learning
• Lack of test standardization among schools and
  districts
• Problems in mechanics of tracking teacher
  candidates and accessing data
• Alternative measures of pupil learning
• Whole-school scores
• Proxies for pupil learning

4
History

• AASCU survey in January 2005
• What campuses were doing to demonstrate the
  effectiveness of their teacher
  education programs.  

• Responses from 240 institutions (65 of
• all AASCU institutions that prepare teachers)
• FTE enrollments approaching
• 2,000,000 students
• Total teacher education degrees
• conferred 65,000

5

• Two basic premises

1) Teacher education accountability is an
important and legitimate goal, particularly for
state institutions that have an obligation to be
accountable to the public.
2) Robust, evidence-based systems demonstrating
effectiveness are needed to achieve educational
outcomes, guide program improvement, and assure
and protect the public.
6
Analysis of Survey Results

• Institutions collect data to assess teacher
  candidates

• Institutions compile voluminous data about
  teacher preparation programs

• Institutions have difficulty accessing
• P-12 pupil achievement data

• Institutions are collecting information that
• is of dubious utility

7
Analysis of Survey Results

• Data collected is all over the map

• Need to focus more on validity
• of data and reliability of evidence

• Energy and resources are going into the
• data collection process

• High level of frustration about data and
• evidence processes

8
Specific concerns about data collection and
analysis

• The amount of time to collect and analyze is
  considerable,
• especially for a small program 2) Different data
  requirements
• and differing definitions to be used in
  collecting and
• summarizing data for all of the formal reporting
  requirements
• of state, federal, and national accreditation
  make analysis
• extremely time consuming and 3) It is a
• continuing challenge to ensure the reliability
• and validity of measures of classroom
• performance by students and graduates.

9
Specific concerns about data collection and
analysis
Major issues include availability of data and
reluctance to release data, even aggregated,
because of union guidelines and confidentiality
issues.
Time, technology and personnel to input,
aggregate, analyze and interpret the volume of
data generated by a large program and
communicating the information in a timely
manner to all who need to have it to make
better decisions.
10
Specific concerns about data collection and
analysis
Database compatibility within and across
organizations program components assessed at
multiple times, by different stakeholders, so
results are not comparable requests, format and
questions worded differently for similar
information participant return rate on surveys
lack of state database support and access.
11
Challenges to developing systems

• Methodological difficulties linking teacher
• practice to pupil outcomes

• Confidentiality and privacy issues in handling
• access to data

• Difficulties in ensuring validity and
  reliability
• of data

• Time and expense required to gather data

• Varying definitions and data requirements
• for reporting

12
Promising National Initiatives

• Measuring Content Knowledge and Teacher
  Performance
• NBPTS
• INTASC
• The Renaissance Group
• Project STEP

13

• Measuring Student Achievement
• Christa McAuliffe Excellence in Teacher
  Education
• Award
• Teachers for a New Era
• Value-added methodologies

14

• Measuring Retention and Support of Teachers
• Data Quality Campaign
• Essential elements and fundamentals of a
• longitudinal data system
• Institute of Educational Sciences
• statewide longitudinal data system
• grants program

15
Promising State and System wide efforts

• Louisiana
• California State University System
• Ohio
• City University of New York System
• Georgia
• South Carolina
• University of Wisconsin System
• CREATE (Texas)

16
A National Framework A set of guidelines that
institutions might use to guide data collection
17
Essential considerations

• Emphasize importance of multiple measures

• Build on promising methods, but allow
• state-by-state adaptation

• Collaborate with all stakeholders

• Develop better state-level data systems

• Assure validity and reliability of data

18
Recommendations