OSPI Mathematics Assessment Update for Grades 38 and High School

1
OSPI Mathematics Assessment Update for Grades 3-8
and High School

• Karen Hall
• 2008 WERA

2
Guidance for Specifications

• RCW 28A.305.215(3)
• The recommendations for revised EALRs and GLEs
  in mathematics shall be based on
• (a) Considerations of clarity, rigor, content,
  depth, coherence from grade to grade,
  specificity, accessibility, and measurability.

3
(No Transcript)
4
Time Line for Grade 3-8 Assessments Aligned to
New Standards

• April 2008 New Grade 3-8 Math Standards approved
• May - November 2008
• Develop Item Specifications for new grade-level
  standards
• Align or revise items in existing test items to
  new grade-level standards
• Write new items to fill in gaps
• April-May 2009
• Last year of current Grade 3-8 math WASL
• Pilot new and rewritten items
• April-May 2010 First administration of new Grade
  3-8 math WASL
• July 2010 State Board adopts new performance
  standards on Gr. 3-8 math tests

5
Time Line for High School Assessments Aligned to
New Standards

• August 2008 New High School Mathematics
  Standards approved
• November 2008 August 2009
• Align existing assessment items to new
  grade-level standards
• Write new items to fill in gaps
• 2009 and 2010 Pilot new and rewritten items
• 2011 First administration of new HS End of
  Course Assessments
• July 2011 State Board adopts new performance
  standards on High School End of Course Assessments

6
Goal 1

• Develop Item Specifications for Grades 3-8 while
  retaining clarity and specificity of Standards
  Document.

6
7
Paragraph Descriptions

• 5.3. Core Content Triangles and
  quadrilaterals (Geometry/Measurement, Algebra)
• Students focus on triangles and quadrilaterals to
  formalize and extend their understanding of these
  geometric shapes. They classify different types
  of triangles and quadrilaterals and develop
  formulas for their areas. In working with these
  formulas, students reinforce an important
  connection between algebra and geometry. They
  explore symmetry of these figures and use what
  they learn about triangles and quadrilaterals to
  solve a variety of problems in geometric
  contexts.

8
Performance Expectation Numbering System

• Grade Level Area of Emphasis Expectation
• 5.3.D

9
Development Task

• Identify which Performance Expectations to assess
  on WASL.

10
Development Tasks

• Identify restrictions, if any, for
• Vocabulary
• Vocabulary First Used in Assessment Items
• Measurement Vocabulary
• Computation
• Number of addends
• Denominators
• Decimal places
• Measurements

11
Development Tasks

• For each Performance Expectation Identify
• Cognitive Complexity (Webbs model)
• Item Type (MC or SA)
• Contextual Situation or not
• Tools or No-Tools day

12
Performance Expectations with Restrictions
13
Stimulus, Stem, and Prompt Rules

• Use Item Development Guidelines at the beginning
  of this document.
• Answer choices will be stated in terms of the
  same system of measurement.
• Items will not require students to convert
  between U.S. customary and metric units.
• Exponents will not be used to express square
  units.
• Items may tell students to use a straight edge or
  a protractor.
• Items will not require use of a particular
  strategy to determine a derived measurement.
• Grids may be provided in items that require
  students to draw angles or figures.

14
Stimulus, Stem, and Prompt Rules

• Items assessing 5.3.A may include parallelograms,
  kites, squares, rhombi, trapezoids, and
  rectangles.
• Items assessing 5.3.C may require triangles to be
  classified by angles as acute, right obtuse or by
  sides as scalene, isosceles, or equilateral.
• Items assessing 5.3.F may include side measures
  or may require students to measure sides of
  figures.

15
Development Process
16
Revisions to Draft Version

• Summer Item Writing
• 120 Item writers, 900 items for grades 3-8
• Excellent feedback on item specifications
• Content Review of New Items
• 5 committees
• 2 weeks

Draft 8
17
Test Experts Recommendation Assign Cognitive
Complexity to each Performance Expectation
18
Why Cognitive Complexity?

• No Child Left Behind (NCLB) requires assessments
  to measure the depth and breadth of the state
  academic content standards for a given grade
  level (U.S. Department of Education, 2003, p.
  12)

19
Why Cognitive Complexity?

• Mechanism to ensure that the intent of the
  standard and the level of student demonstration
  required by that standard matches the assessment
  items (required under NCLB)
• Provides cognitive processing ceiling for item
  development

20
Alignment
Standards
Standards
Assessment
Assess-ment
Assessment Items
Assessment
Standards
Adapted from Norman Webb, 2005
21
Develop assessment items to align with cognitive
complexity of Performance Expectation
22
Mathematical Complexity of ItemsNAEP 2005
Framework

• The demand on thinking the items requires
• Low Complexity
• Relies heavily on the recall and recognition of
  previously learned concepts and principles.
• Moderate Complexity
• Involves more flexibility of thinking and choice
  among alternatives than do those in the
  low-complexity category.
• High Complexity
• Places heavy demands on students, who must
  engage in more abstract reasoning, planning,
  analysis, judgment, and creative thought.

23
(No Transcript)
24
Depth of Knowledge (DOK)

• Adapted from the model used by Norman Webb,
  University of Wisconsin, to align standards with
  assessments
• Used by the Council of Chief State School
  Officers (CCSSO) for assessment alignment in more
  than ten states

25

• The Depth of Knowledge is NOT determined by the
  verb, but the context in which the verb is used
  and the depth of thinking required.

26
Depth of Knowledge (DOK)

• Level 1 (Low) Recall
• Level 2 (Moderate) Skill/Concept
• Level 3 (High) Strategic Thinking
• Level 4 Extended Thinking

NOTE Definitions with examples of skills at each
level.
27
Recall and Reproduction Level 1
DOK 1 requires recall of information, such as a
fact, definition, term, or performance of a
simple process or procedure. Answering a Level 1
item can involve following a simple, well-known
procedure or formula. Simple skills and abilities
or recall characterize DOK 1.
28
Skills/Concepts Level 2
DOK 2 includes the engagement of some mental
processing beyond recalling or reproducing a
response. Items require students to make some
decisions as to how to approach the question or
problem. These actions imply more than one
mental or cognitive process/step.
29
Strategic Thinking Level 3
DOK 3 requires deep understanding as exhibited
through planning, using evidence, and more
demanding cognitive reasoning. The cognitive
demands at Level 3 are complex and abstract. An
assessment item that has more than one possible
answer and requires students to justify the
response they give would most likely be a Level
3.
30
Extended Thinking Level 4
DOK 4 requires high cognitive demand and is very
complex. Students are expected to make
connections and relate ideas within the content
or among content areasand have to select or
devise one approach among many alternatives on
how the situation can be solved. Due to the
complexity of cognitive demand, DOK 4 often
requires an extended period of time.
31
However, extended time alone is not the
distinguishing factor.
32
Remember

• Depth of Knowledge (DOK) is a scale of cognitive
  demand.
• DOK requires looking at the assessment
  item/standard order to determine the level. DOK
  is about the item/standard-not the student.
• The context of the assessment item/standard must
  be considered to determine the DOK-not just a
  look at which verb was used.

33

• Read example items and discuss cognitive
  complexity of each item

34
Develop Grades 3-8Test Maps
35
Test Map (Blueprint)

• Total Number of points
• Total Number of Multiple-Choice Items
• Total Number of Short-Answer Items
• Distribution of Points by Area of Emphasis
• Distribution of Points by Cog. Complexity

36
What we know!

• Grades 3-5

• Grades 6-8

• 38 Points Total
• 24 Multiple-Choice
• 7 Short-Answer

• 50 Points Total
• 30 Multiple-Choice
• 10 Short-Answer

37
Incomplete Test Map Grade 5
38
Develop a Draft Test Map

• Surveyed 150 teachers, coaches and mathematics
  coordinators
• Workgroup to distribute test points among areas
  of emphasis
• Cathy Seeley, Dana Center
• George Bright, OSPI, Mathematics Special
  Assistant
• OSPI Teaching and Learning Mathematics
• OSPI Mathematics Assessment
• Test Map Review Committee
• Standards Revision Team
• Teachers, Coaches, Administrators

39
Draft Test Maps

• Survey results
• Paragraphs in Standards Document
• Performance Expectations

40
40 Grade 5 Performance Expectations

• 34 BOLD text Performance Expectations
• 31 test questions
• 38 total points.

41
Next Steps

• Use results of test map review to finalize test
  maps for grades 3-8
• Submit test maps for approval to National
  Technical Advisory Committee on January 11, 2009
• Post Grades 3-8 Test and Item Specification
  Document on OSPI website.
• Review and feedback of assessment restrictions
  for high school performance expectations
  January Conference
• Write items to assess high school PEs

42
Performance Expectation InformationGrades 3-8