Public Indicators of Public Education in the U.S.

1

• Public Indicators of Public Education in the U.S.
• Andrew W. Shouse
• National Research Council
• National Academies, U.S.A.
• www.nas.edu/bose
• Ashouse_at_nas.edu

2
Three Goals

• Characterize the political and policy context of
  educational indicators in the U.S.
• Describe the technical capacity of NAEP to assess
  student inquiry
• Make the case for indicators that target a
  different public

3
U.S. Education Governance

• Decentralized System
• Federal, State, Local
• With Centralizing Tendencies
• No Child Left Behind Federal Policy

• Decentralized System

• Decentralized System

4

• Many Indicators
• Teacher Quality
• Credentials, Teacher Exams, In/Out of Field
• Student Assessments
• Federal, State, Local
• Many Purposes
• School/System Improvement
• Accountability (sanctions rewards)
• Teachers, Schools, Districts, States
• School Choice
• Blue Ribbon or Blacklist
• Science Academies, Other Specializations

5
Many Educators Perceive Indicators as Tools of
Punishment
6

• Goal 2Describe the technical capacity of NAEP to
  assess inquiry knowledge and skill

7
Source Senta Raizens address to the Committee
on Science Learning, K-8. March 2005
8
Characteristics of Main Science NAEP

• Large-scale, nationally-representative
  assessment at grades 4, 8, 12 not a test for
  individual students
• Science given every four years

9
What NAEP Can Do Well

• NAEP commands national notice for its
• role in monitoring student achievement
• methodology, as a model for assessment
• focus on standards-based reporting
• NAEP frameworks are a resource for international,
  state, and local curriculum and assessment
• NAEP can be used for research

10
Assessing Inquiry

• Find patterns in observations and constructing
  explanations for those patterns.
• Design scientific observations and
  investigations.
• Conduct scientific observations and
  investigations.
• Use appropriate tools and techniques to gather,
  analyze, and interpret data.
• Think critically and logically to identify the
  relationships between evidence and explanations.
• Raise questions about and criticize important
  components of inquiry conducted by self and
  others (including inquiry results and conclusions
  reported in the media).

11
Assessing Inquiry

• Using scientific patterns and models to explain,
  predict, design
• Use patterns or models to explain observations
• Use patterns or models to predict observations
  (e.g., solving word problems, qualitative
  predictions)
• Recognize and analyze alternative explanations
• Communicate procedures and explanations

12
Limitations

• To what extent, if at all, can NAEP assess
• Sustained work
• Meeting unpredictable challenges
• Altering approach based on evidence collected
• Deliberation and reasoning to modify question
• Team work
• Engaging with fellow students and/or teacher

13
Hands-On Investigations

• Benefits
• More authentic
• Closer to classroom experiences
• Testing relevant skills
• Engaging
• Sends message about importance of doing
  science

• Challenges
• Takes time
• Standardization for reliable scoring is hard
• Equipment costs/reliability
• Scoring costs
• Content limits

14

• Excellent inquiry frameworks measurement
  technologies exist.
• These (and there are others) should be considered
  and improved upon.

15

• Goal 3 Make the case for indicators that target
  a different public

16
How well can state and national assessments of
student learning influence science teaching?
science
17
Key Members of the IBSE Public52 million
students, 2.2 million teachers!
science
18
Public Indicators That Target Teachers and
Learners

• Formative Assessment Practices Use of any
  student performance data to inform instruction
• System Level
• Curriculum Embedded Benchmark Assessments
• Link Measures of Student Learning with Specific
  
• Teaching Strategies
• Classroom Level
• Questions Tasks Assess Student Understanding
  Skill

19
Technical Rationale

• Formative assessment speaks to people who have
  control over instruction teachers students
• Formative assessment is an integral part of IBSE
• Most formative assessment practices can be
  conducted in the flow of instruction

20
Empirical Rationale

• Formative assessment practices are consistently
  associated with student learning benefits (see
  reviews by CPRE, Black Wiliam and others)

21
Political Rationale

• Policymakers want evidence-based practice
• Teachers want practical support that works
• Everybody wants improved student learning