Assessing Your Standards: Standards and Assessment Alignment

1
Assessing Your Standards Standards and
Assessment Alignment
Adult Education Content Standards Consortia
Meeting

• Regie Stites
• July 22, 2005
• U.S. Department of Education, Division of Adult
  Education and Literacy
• American Institutes for Research

2
Assessment is ?
3
Assessment is

1. Rocket science
2. A critical component of effective teaching
3. Not all that it needs to be to support
   standards-based educational improvement

4
True or False

• A test is a test is a test
• A score is a score is a score
• Two tests that measure the same thing can be
  equated
• Multiple choice questions only measure recall
• You can tell a good item by looking at it
• Multiple-choice test Standardized test
  High-stake test

5
Assessment Literacy

• All educational assessments are observations and
  interpretations of defined constructs (models of
  cognition)
• (A)n assessment result is an estimate, based on
  samples of knowledge and performance from the
  much larger universe of everything that a person
  knows and can do (from Knowing What Students
  Know The Science and Design of Educational
  Assessment, NRC, 2001, p. 37)

6
Best Assessment Practices

• Assessments should be used for the purpose they
  were designed to serve for different purposes
  use different tests
• Purpose is defined by when you give the test
  (before, during, or after instruction) and by how
  you will use the test results (formative or
  summative)

7
Formative Assessment

• Key purpose is to use test results (usually
  before or during instruction) to plan, monitor,
  and improve teaching and learning
• Should be curriculum-based and aligned with
  standards
• May or may not be standardized
• Can be selected or locally-developed

8
Summative Assessment

• Key purpose is to use results as evidence of
  achievement (usually pre/post or post
  instruction)
• May or may not be high-stakes for the learner
• Should be a standardized test to ensure
  reliability and fairness
• Can be used as partial evidence of instructional
  quality (and to guide improvement)

9
Seeing Standardized Tests In A New Light

• Recognize their true utility dont ignore
  results and dont over-interpret them
• Be patient - help everyone understand that
  standardized test results are just one of many
  sources of evidence of learning on standards
• Question the experts ask for the help you need
  to improve alignment of standards and assessments

10
Webbs Criteria

• Categorical concurrence
• Extent to which assessment items address same
  content as standards
• Depth-of-knowledge consistency
• Extent to which cognitive complexity of responses
  to assessment tasks matches what learners should
  know and be able to do as stated in the standard
• Range-of-knowledge correspondence
• Proportion of indicators of standards covered by
  assessment items
• Balance of representation
• Extent to which assessment items are distributed
  evenly across indicators

11
Alignment Resources

• Norman L. Webb (1999). Alignment of Science and
  Mathematical Standards and Assessments in Four
  States
• http//www.ccsso.org/content/pdfs/AlignmentPaper.
  pdf
• CCSSO Alignment Analysis Web page
• http//www.ccsso.org/Projects/Alignment_Analysis/
  

12
Guided Practice 1

1. Select high priority standards that are most
   suitable for standardized testing (with the test
   you have)
2. With the Webb criteria in mind evaluate the
   likely alignment of these standards to your
   current test and options for additional
   assessment if needed

13
Guided Practice 2

1. Discuss the need to organize an assessment review
   or a formal alignment analysis of your state
   standards and the test you use for program
   accountability
2. Map out plans for a review of assessment options
   or a formal alignment analysis