Creating Assessments to Build Teacher Capacity to Inform the Instruction/Learning of EACH Student

1
Creating Assessments to Build Teacher Capacity
to Inform the Instruction/Learning of EACH Student

• Prepared for the Instructional Team of Jamestown
  High School by
• Dan Mulligan
• Simply Achieve, Inc.
• November 2006

2
Required pass rates for each subgroup 2005
tests reading (65)
math (63) 2006 tests reading (69) math
(67) 2007 tests reading (73) math
(71)
3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
Depth of Knowledge levels Based on Blooms
Taxonomy

• Level 4 Higher-Order Thinking
• Combines the three most complex levels of
  cognitive processes
• Analysis standards or test items would require
  students to break down information into parts to
  categorize, diagram, differentiate, discriminate,
  outline, separate, or subdivide content.
• Synthesis students would be expected to
  combine elements into a whole to integrate,
  organize construct, design, combine, arrange,
  compile, create formulate, generate, propose,
  project, integrate.
• Evaluation students would be asked to judge,
  assess, appraise, argue, verify, support, defend,
  evaluate, recommend.

8
Earth Science
9
Depth of Knowledge levels Based on Blooms
Taxonomy

• Level 3 Application
• Involves the process of using known information
  to solve problems.
• includes using abstract ideas, rules, or
  generalized methods in new concrete situations
• Application-level standards or test items may
  require students to apply, construct,
  demonstrate, illustrate, modify, produce, show,
  solve or use.

10
Algebra I
11
Depth of Knowledge levels Based on Blooms
Taxonomy

• Level 2 Comprehension
• Involves the understanding or grasping the
  meaning of a concept or information (e.g.,
  students explaining an idea or concept in ones
  own words.)
• Comprehension-level standards or test items may
  require students to translate, rephrase,
  interpret, describe, classify, compare, contrast,
  estimate, generalize, explain, give examples,
  infer, interpret or summarize.

12
Grade 9 English
13
Depth of Knowledge levels Based on Blooms
Taxonomy

• Level 1 Recall Knowledge
• Involves recalling previously learned material
  such as facts, events, persons, dates, methods,
  procedures, concepts, principles, and theories.
• Comprehension-level standards or test items may
  require students to count, define, label list,
  match, name, quote, recite, repeat, reproduce,
  select or sate content information.

14
World History II
15

• If an educator keeps using the same strategies
  over and over and the student keeps failing,

who really is the slow learner?
16
Standards Verbs

• PROBLEM SOLVING
• Analyze Derive Discover Evaluate Explore
• Predict Solve Survey Verify Investigate
• REASONING
• Categorize Classify Compare Contrast Differentiat
  e
• Describe Estimate Explain Generalize Interpret
• Justify Order Hypothesize Predict Infer
• Prioritize Rank Validate Summarize
• COMMUNICATION
• Clarify Correspond Describe Discuss Demonstrate
• Exhibit Explain Express Persuade Portray
• Restate Show Speak State Write

17
Analysis of Cognitive Skills Distribution of
Cognitive Skills in Tasks Required of 3rd and
5th Grade Students in Virginia (in percentage per
subject)
Source document Virginia Enhanced Scope and
Sequence documents
18
Distribution of Cognitive Skills in Required of
Grade Students in the Virginia Grade 5 Science
SOL
Source document Virginia Curriculum Framework
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
Virginia SOL Mathematics Test Blueprint Summary
Table
30
(No Transcript)
31
Types of Assessment Items and Formats Related to
Different Aspects of Learning
Assessments
Transforming Classroom Grading, ASCD, 2002
Note L Low, M Medium, H - High
32

• Aligned Assessment Items are
• Grounded in a specific essential knowledge
  skill
• Formatted in the style of an SOL test item
• Key words italicized
• Numeric choices are in ascending/descending order
• Completion questions use a raised dash

33

• Aligned Assessment Items are
• Identified by creating a cataloging system
• SOL
• Bullet from SOL
• Bullet in Essential Knowledge Skills

7.8b
34

• Aligned Assessment Items are
• Identified by creating a cataloging system
• SOL
• Bullet from SOL
• Bullet in Essential Knowledge Skills

8.12a
35
(No Transcript)
36
Designing High-Quality Paper-and-Pencil Tests

• Parts of a Multiple-Choice Item
• Directions
• Visual
• Item Stem
• Response
• Options
• Stimulus

• Use the picture below to answer the following
  question.
• About how many paper clips long is this pencil.
• A 2 paper clips
• B 3 paper clips
• C 4 paper clips
• D 5 paper clips

37
Assessment Informs Instruction
Instruction
Assessment
Curricula
38
Principles for Writing Quality Multiple Choice
Items

• Align and organize your assessment items with the
  state standards by -
• Identifying both the CONTENT and the LEVEL OF
  THINKING that your item will address.
• Applying the Principles for Developing Quality
• Items Stems
• Response Options
• Stem Starters
• Visuals in Test Items
• Item Format to Align with State Assessments

39
Principles for Writing Quality Item Stems

• Write the item stem in one of two ways.
• As a good, solid question
• As a statement, that when matched with the
  correct answer, is a complete thought
• Overall, state item stems positively.
• However, students need practice with items that
  use negative words (e.g. NOT, LEAST, IS NOT,
  EXCEPT, NOT TRUE). These words should be
  emphasized using bold, italics, underlining, or
  CAPITAL lettering.

40
Principles for Writing Quality Item Stems

• Highlight key words.
• Highlight key words by using italics
• Examples of key words include BEST, ALL, NONE,
  EXCEPT, GREATEST VALUE, IS, IS NOT, LEAST
  ACCURATE, MOST VALUABLE, and MOST ACCURATE.
• Stuff the stem.
• Include most of the information for your question
  in the stem and not in the responses.

41
Principles for Writing Quality Response Options

• Provide brief responses.
• Have one correct answer.
• Write plausible distractors.
• Arrange numbers in ascending or descending order.
• Write the answer choices so they are
  grammatically consistent with the stem.
• Make sure the right answer is always there.
• Watch trick key words. Examples include ALL,
  ALWAYS, ENTIRE, EVERY, NEVER, NONE, ONLY,
  COMPLETELY.
• Arrange answer choices vertically.

42
Principles for Using Quality Visuals

• Include visuals on some of your test items.
• Use a variety of visual representations
• Make sure each visual representation has a
  purpose.
• Place visuals in different parts of the item.
• Make sure visuals are clearly reproduced and easy
  to read.
• Use more than one question for complex graphics.
• Place questions related to the visual on the same
  page or facing pages.

43
(No Transcript)
44
http//www.childrenslibrary.orgwww.ttaconline.or
g