Evaluation of the NASA Aerospace Education Services Program

1
Evaluation of the NASA Aerospace Education
Services Program

• Presentation to the National Academies National
  Research Council
• Board on Science Education
• January 19, 2007
• Kenneth H. McKinley

2
Background of the Study

• Contract between The Evaluation Center, Western
  Michigan University and NASA-AESP
• 3-year Project November 2001 October 2004
• Studied process and program only-NOT budget or
  cost-effectiveness.

3
Purpose and Scope of the Study

• Evaluation of five basic areas
• Program design and management
• Support of systemic improvement in NASA Education
• Teacher preparation and enhancement of programs
  that support systemic reforms
• Student support
• Curriculum and dissemination

4
AESP Historical Perspective

• 1962 93 Space-mobile program
• 1994 2002 State standards, curriculum
  frameworks, testing
• 2003 - ? NASA Explorer Schools

5
Evaluation Philosophical Approach

• Context, Input, Process, Product (CIPP) Model
• The Program Evaluation Standards (Joint Committee
  on Standards)

6
Study Methodology, Instruments

• Surveys
• Interviews
• Observations
• Document reviews
• Focus group meetings
• Data base examination

7
Evaluation Challenges/Support

• Measuring impact
• Time challenges
• Unique, but extensive, vocabulary
• Excellent cooperation from NASA HQ, Centers,
  Contractor (OSU).

8
Evaluation Outcomes/Findings

• Program Design and Management
• Cooperative agreement facilitates flexibility,
  field responsiveness
• Unique AESP command and control management
  structure
• Can-do attitude within ranks

9
Evaluation Outcomes/Findings

• Program design (contd)
• Current AESP workforce time allocation
• 60 - Explorer Schools
• 20 - Non-Explorer Schools
• 10 - Distance Learning Network involvement
• 10 - On-Center programming

10
Evaluation Outcomes/Findings

• Support of Systemic Reform
• Limited incorporation of space science in state
  standards
• Many states already had standards well-developed
• Changing priorities in NASA-ED.
• New administrator
• New program initiatives

11
Evaluation Outcomes/Findings

• Teacher Preparation and Enhancement
• Low priority low time investment by AESP
• Negligible impact on classroom practice
• AESs help clarify science content and teacher
  conceptual understanding.

12
Evaluation Outcomes/Findings

• Student support
• Limited impact on pre-service teacher preparation
• Pre-service teacher education a fairly closed
  system.

13
Evaluation Outcomes/Findings

• Curriculum and dissemination
• Strong awareness of NASA ED message provided to
  educator, public groups.
• Service to informal education relatively minor
• Museums, planetariums, learned societies

14
Final Summary Points to Ponder

• AESP is NASAs face-in-the-field
• Excellent support system for other NASA ED
  pre-college initiatives
• High quality, flexible workforce
• Strong service delivery culture
• NASA Center specialist domicile/single management
  contractor (university) structure is good
• No one statement defines AESP.