2005 NAEP Release Activities

1

• 2005 NAEP Release Activities
• Arnold Goldstein, NCES

2
Upcoming 2005 Releases
3
State Pre-Release Workshop

• Held Sept. 27-29
• Preview of national results
• Access to state Snapshot report
• Access to web tools State Report Generator,
  NAEP Data Explorer, NAEP Questions Tool
• Sessions on interpreting results, frameworks,
  scoring, working with the media, using items to
  illustrate performance, communicating about NAEP
  and state assessments

4
Access to Embargoed Data

• States will have access to their embargoed NAEP
  data prior to the release
• State Report Generator
• NAEP Data Explorer
• NAEP Questions Tool
• State Snapshot Report
• Materials and data collected from states at the
  end of the Pre-Release Workshop will be sent to
  the Coordinators

5
TUDA Pre-Release Workshop

• Held November 2-3
• Preview of TUDA results
• Access to district Snapshot report
• Access to web tools NAEP Data Explorer, NAEP
  Questions Tool
• Sessions on interpreting results, frameworks,
  scoring

6
Trial Urban District Assessment
7
Access to Embargoed Data

• Districts will have access to their embargoed
  NAEP data prior to the release
• State Report Generator
• NAEP Data Explorer
• NAEP Questions Tool
• State Snapshot Report
• Materials and data collected from districts at
  the end of the Pre-Release Workshop will be sent
  to the TUDA contacts

8
Planned NAGB Briefings
9
Plans for the Day of the Release

• Initial Release Website available
• Live web cast of the press conference
• Reports posted on the NAEP website
• NAEP Data Explorer populated with 2005 data
• Stat Chat online QA

10
National Assessment of Adult Literacy (NAAL)
Release

• Two upcoming publications
• A First Look at the Literacy of Americas Adults
  in the 21st Century
• 24-page overview highlighting basic findings,
  with an emphasis on adults with Below Basic
  literacy
• Key Concepts and Features of the 2003 National
  Assessment of Adult Literacy
• Covers critical features and concepts carried
  over from 1992 assessment as well as those new to
  the 2003 assessment

11
Planned NAAL Release Activities

• Press conference
• Live web cast
• New NAAL web site http//nces.ed.gov/naal/
• Reports posted on the NAAL website
• Stat Chat online QA

12

• Questions?

13

• NAEP Update
• Peggy Carr, NCES

14

• NCES Update

15
Commissioner Nominee

• Presidential nominee for Commissioner of
  Education Statistics Mark Schneider
• Current Deputy Commissioner of the National
  Center for Education Research (NCES)
• Former Professor and Chairman of the Dept. of
  Political Science at the SUNY at Stony Brook
• Former Vice President of the American Political
  Science Association
• Term would expire June 20, 2009

16

• Pending NAEP Legislation

17
S 860 (American History Achievement Act)

• Reintroduced by Sen. Ted Kennedy and Sen. Lamar
  Alexander
• Would require trial NAEP state assessments in
  U.S. history and civics (grades 8 and 12)
• Trial in 10 states selected by NAGB and NCES
• Hearings held in Health, Education, Labor and
  Pensions (HELP) Committee

18
S 1089 (National Foreign Language Coordination
Act of 2005)

• Introduced by Sen. Daniel Akaka, Sen. Christopher
  Dodd, and Sen. Thad Cochran
• Would establish the National Foreign Language
  Coordination Council to develop and implement a
  foreign language strategy (to increase foreign
  language speakers in the U.S.)
• Would provide assistance for foreign language
  assessments in NAEP

19
HR 770 (NAEP to be subject to certain guidance on
the release of information to the public)

• Introduced by Rep. Carolyn Maloney and Rep.
  William Lacy Clay
• NAEP would be subject to Statistical Policy
  Directive on Compilation, Release, and Evaluation
  of Principal Federal Economic Indicators
• Would affect NAEPs release schedule and
  pre-release activities
• Last report of each calendar year must contain
  the time and date of all reports in the upcoming
  year. Each release would have to include an
  announcement of the time and date of the next
  release.
• Pre-release access cannot precede the official
  release time by more than 30 minutes.

20

• NAEP Studies

21
Secondary School Principals Working Group on 12th
Grade Participation in NAEP

• Purpose To get input from secondary school
  principals with experience administering NAEP to
  high school students in order to address issues
  related to
• 12th grade participation in NAEP
• The unique logistical challenges that come with
  administering NAEP in high schools
• Improving communication and working relationships
  with principals and high schools

22
Principals

• Tim Davis, Central High School, Rutherfordton, NC
  
• Todd Harvey, Glasgow High School, Newark, DE
• Charles Heinlein, St. Mary's High School, St.
  Mary's, WV
• Jean Massey, Northwest Rankin High School,
  Brandon, MS
• Keith Maxey, Blue Springs South High School, Blue
  Springs, MO
• John Modest, West Charlotte High School,
  Charlotte, NC
• Linda S. Perie (retired principal), Snoqualmie,
  WA
• Oscar Troncoso, Socorro High School, El Paso, TX
• Don Wood-Foucar, Helena High School, Helena, MT

23
Meetings

• Convened in August to orient the members to NAEP
  and to start thinking about the issues
• Reconvened in September to discuss issues
• Additional principals from non-participating
  schools and schools with low student
  participation attended to provide insights

24
Panel Discussion Issues

• The sample should be selected by the early spring
  of the previous year so that NAEP can be added to
  the schools calendar
• Schools and students should get feedback on
  achievement
• 12th grade NAEP should take place in the fall
• 12th grade NAEP should be conducted at the
  state-level

25
NAEP State Analysis Project

• Purposes
• Integrate state NAEP and state assessment results
  into coherent portrayals of reading and
  mathematics achievement trends
• Provide information about factors that affect the
  interpretation of state NAEP results
• Clarify the stories that state NAEP assessments
  tell about achievement trends

26
Activities to Date

• Comparison of NAEP and States assessment results
• Database on State policies and practices for
  Students with Disabilities and English Language
  Learners
• Study on NAEP vs. States assessment
  methodologies

27
State Analysis Requests

• Consortium of states can request special analyses
  of NAEP and state data
• Purpose Develop a model and/or methodology for
  all states to apply as needed
• Request must be for a model/methodology that
  could be generalized across states
• Requests should be made through the NAEP State
  Service Center
• State Assessment Directors should sign-off on the
  request

28

• Questions?

29

• 2006 NAEP Activities
• Peggy Carr, NCES

30

• Assessment Activities

31
Changes due to Hurricane Katrina

• Special study on 12th grade fall testing (a.k.a.
  Participation Engagement study) postponed to
  fall 2006 and winter 2007
• Areas with worst impact not to be included in the
  2006 national assessments
• Potential for other participation issues emerging
  out of national disasters

32
Current Plans for 2006

• National Assessments
• U.S. History (grades 4, 8, and 12)
• Civics (grades 4, 8, and 12)
• Economics (grade 12)
• Field Tests
• Reading (grades 4 and 8)
• Mathematics (grades 4 and 8)
• Writing (grades 8 and 12)

33
Sensitivity to Instruction study

• Purpose investigate the extent to which NAEP
  measures learning gains facilitated by an 8th
  grade reform-based mathematics curriculum in
  comparison to an assessment that is compatible
  with reformed-based approaches and the respective
  state assessment.
• Conducted in Arkansas, Colorado, Oregon, and
  Texas
• Baseline testing complete
• Post-testing planned for summer 2006
• Analysis and report planned for winter 2007

34
2007 Assessments (1)
35
2007 Assessments (2)

• Letters regarding 2007 assessments sent to Chiefs
  and Assessment Directors in early 2006
• Copy to NAEP State Coordinators
• Includes participation agreement for Writing

36

• Evaluation Activities

37
Congressionally-mandated review of NAEP (1)

• Law requires an external, independent evaluation
  of the overall NAEP program
• Overseen by U.S. Dept. of Educations Policy and
  Program Studies Service
• Carried out by the Buros Center for Testing
  (Univ. of Nebraska)

38
Congressionally-mandated review of NAEP (2)

• Buros reviewing all NAEP functions
• Examples development of items and background
  questions, setting of achievement level
  standards, sampling, scoring
• Site visits with NAEP contractors including
• Westat (to review sampling and data collection
  and the NAEP State Service Center)
• Meeting with 6 NAEP State Coordinators

39
NAEP State Coordinator Program Review

• Purpose to examine what is being accomplished,
  whether the goals for the program are being met,
  and what actions or changes NCES needs to
  consider
• One-day visits with the Coordinator, Assessment
  Director, and others who are/have been involved
  in NAEP
• 5 states visited between July and September
• Plans for every state to be visited within 3
  years

40

• Questions?