[e-mail]. 3. Today's Questions. Why test

1

• Those who can do.
• Those who understand teach.
• Those who teach assess.

2
Choosing Test ItemsThere are no perfect items.

• Regional Collaboratives
• February 12, 2007
• presented by
• Virginia Malone
• Virginia_Malone_at_juno.com

3
Todays Questions

• Why test?
• What is being tested?
• What makes some test items better than others?

4
Why Test?
5
(No Transcript)
6

• It is kind of nice to know why you do what you do.

7
Major Reasons for Testing

• Diagnostic
• Comparative
• To standards (criteria base)
• To others (norm reference)

8
What is being tested?
9
What do we want to learn from the test?

• Who knows the science
• and who does not.
• NOT who is a good reader and who is not
• NOT who a good test taker and who is not

10
Need Test Blueprint

• Assure that what should be taught is tested.
• Assure a balanced coverage of Performance
  Objectives.

11
Test Blueprint

• Define what tests will cover.
• Guide for reporting.
• Keep test forms parallel.
• Pre/post tests

12
What will be reported?

• Strands
• Concepts
• Performance Objectives
• Vocabulary
• 5 items for percent correct score
• 10 items for scaled scores

13
One Type of Blueprint
14
What makes some test items better than others?

• There is no such thing as a perfect item.

15
Thought Provoking Test Items

• Require the use of both knowledge and skills.
• May cover more than one TEKS.
• Go beyond reading and recalling.

16
The ixleb was heb. That narf, the illby ixleb
summit and was all domini. Where was its zomba?
The ixleb zanked and then izzly zonched for its
zomba from the hightow. It zonched because an
uga of dow filled that narf er - dow of the
dumbing tonto's bleer. 1. What was the ixleb? 2.
When did the illby ixleb summit? 3. Who was the
illby? 4. Where did the ixleb zank and zonch
from? 5. Why did the ixleb izzly zonch?
17
Appropriate Reading Level

• Reading levels as low as possible.
• Sentence structure simple.
• Words other than science should be as simple as
  possible.
• Short sentences rather than long.
• Avoid there

18
More than Rote Memory

• poor
• Which of these is present in eukaryotic cells,
  but not prokarotic cells?
• A. a cytoplasmic membrane
• B. a chromosome
• C. a nucleous
• D. a cell wall

19
Application to new situation

• better
• A cell is isolated that contains a cytoplasmic
  fluid phospholipid membrane, a cell wall, and a
  single circular chromosome. This cell most
  likely belongs to which type of organism?
• A. a plant
• B. a prokaryote
• C. a protozoan
• D. a fungus

20
Recall

• Objective Identify the six most common elements
  in living things.
• poor
• Which of these is the most common element in
  living things?
• Iron
• Sulfur
• Calcium
• Carbon

21

• better but lots of reading
• Some students saw some white salt (NaCl)
  crystals at the edge of a drop of water. As the
  water evaporated the salt crystals became bigger
  and more of them appeared in the dish. All of the
  following would seem to indicate that the
  crystals are alive EXCEPT the fact that the
  crystals --
• grow in size.
• lack hydrogen.
• appear to reproduce.
• contain chlorine.

22

• Graphics can simplify and engage.
• The diagram shows salt crystals that
  appeared in a dish after salt water evaporated.
  Which of these shows that the salt is not alive?
• The size of the crystals increased.
• They lack hydrogen.
• C. They appear to reproduce.
• D. They contain chlorine.

23
Use graphics to simplify not to decorate.

• Poor
• The picture shows a mouse. Mice have fur and
  feed their young milk. Mice belong with what
  group?
• A Mammals B Insects
• C Fish D Birds

24
Use graphics to simplify not to decorate.

• Better
• Mice have fur and feed their young milk. Mice
  belong with what group?
• A Mammals
• B Insects
• C Fish
• D Birds
• Grade level and ELL considerations

25
Alignment to Objectives

• Objective Understand the transfer of heat.
• poor
• Which temperature is equivalent
• of 273 K?
• A. 0 oC B. 32 oF
• C. 100 oC D. - 273 oC

26
Objective Understand the transfer of heat.

• better
• Which diagram correctly shows the direction of
  net heat transfer?

27
Avoid tricky questions or two justifiable correct
choices

• poor
• Which of these shows the greatest evidence of
  erosion?
• A mountain range
• A canyon
• A delta
• A meteor crater

28

• better
• Which of these landforms is the result of
  erosion caused by primarily by water?
• A A volcano
• B A canyon
• C A meteor crater
• D A sand dune

29
Avoid the Use of You

• poor
• Which of these would you do if a student asked
  you to help them on a test?
• A. Reprimand the student.
• B. Try to help the student without giving
• away the answer.
• C. Ask the student to stay after school
• for more help
• D. Remind the student that you are
• not allowed to help on the test.

30

• Better
• According to the directions for administering
  the test, which of these should a teacher do if a
  student asks for help on the test?
• A. Reprimand the student.
• B. Try to help the student without giving away
  the answer.
• C. Ask the student to stay after school for more
  help
• D. Remind the student that you are not allowed
  to help on the test.

31
Items should provide focus.

• Poor
• The earth is
• A. spherical more or less
• B. made up of elements
• C. part of the solar system
• the third rock from the sun

32
Providing Focus Makes It Clear What a Student Is
Being Asked to Do.

• better
• Which of these best describes the shape of
  the earth?
• A. An oblate spheroid
• B. A circle
• C. A torus
• D. A lumpy sphere

33
Items should avoid the use of negatives.

• poor
• Which of these is not a characteristic of
  animals?
• A. They eat other organisms.
• B. They have no chlorophyll.
• C. They do not make their own food.
• D. They have cell walls.

34

• better
• Which of these is a characteristic that all
  animals have?
• A. Four legs
• B. Food getting behavior
• C. Chlorophyll
• Blood

35
Items should not clue other items

• Vitamin B, found in rice hulls, helps prevent--
• A. beriberi
• B. scurvy
• C. measles
• D. botulism
• As westerners introduced better milling
  techniques to remove rice hulls, natives of south
  sea islands began to develop beriberi due to a
  lack of --
• A. calcium
• B. a vitamin
• C. citric acid
• D. protein

36
Items should not require knowledge of the correct
answer to a previous question.
1. At what point on this diagram is winter in
North America represented?
2. During what season in North America is the sun
closest to Earth?
37
Options should be in logical order.

• Which seeds are probably the slowest to
  germinate?
• Mango B. Tomato
• C. Peach D. Watermelon

Which seeds are probably the slowest to
germinate? A. Watermelon B. Tomato C. Mango
D. Peach
38
Rationales for distractors.

• What is the greatest percent of weed control
  in this investigation?
• A. 55 the lowest value shown
• B. 85 the greatest value for single
  treatment
• C. 90 The greatest value
• D. 200 Just a number no justifiable
  rationale

39
The question must have one clearly correct
answer.

• Tundras and taigas are found at high latitudes,
  but similar biomes can also be found
• A. at high altitudes
• B. at high longitude
• C. along coastlines
• D. in the interior of continents

40
The options should be free of repetitious
information.

• poor
• When is the earth farthest from the sun?
• A. During the spring in the Northern Hemisphere
• B. During the summer in the Northern Hemisphere
• C. During the fall in the Northern Hemisphere
• D. During the winter in the Northern Hemisphere

41

• better
• During what season in North America is Earth
  farthest from the sun?
• A. Spring
• B. Summer
• C. Winter
• D. Fall

42
When possible, items should be complete questions.

• poor
• The test item type that is most efficient and
  reliable is the
• A. Multiple Choice
• B. Constructed Response
• C. Matching
• D. True/false

43
Complete Questions

• better
• Which of these is the most efficient and
  reliable type of test item on a 30 item test?
• A. Multiple Choice
• B. Constructed Response
• C. Matching
• D. True/false

44
Distractors should be plausible.

• poor
• Which of these is a planet in our solar
  system?
• A. Water
• B. Mars
• C. An asteroid
• D. Dirt
• No real rationale for answers

45
Plausible distractors

• better
• Which of these is a planet in our solar
  system?
• A. The sun
• B. The moon
• C. The planet Mars
• D. The comet Halley

46
Options parallel in structure and degree of
specificity.

• poor
• Which of the following is a compound?
• A. Kinetic energy
• B. Wind that is blowing
• C. Pure distilled water
• D. A soft drink like Pepsi

47
Parallel Options

• better
• Which of the following is a compound?
• A. Sulfur
• B. Iron
• C. Water
• D. Hydrogen

48
The options should be free of repetitious
information.

• poor
• When is the earth farthest from the sun?
• A. During the spring in the Northern Hemisphere
• B. During the summer in the Northern Hemisphere
• C. During the fall in the Northern Hemisphere
• D. During the winter in the Northern Hemisphere

49

• better
• During what season in North America is the
  earth farthest from the sun?
• A. Spring
• B. Summer
• C. Winter
• D. Fall

50
Items should not contain options that are
subsumed by other options.

• poor
• A patient had a high fever, a greater blood
  pressure on the right side of his body than his
  left. He was loosing about one pound a day.
  Which of these caused this person to become ill?
  
• A. HIV
• B. vasculitis
• C. circulatory illness
• an infection

51
Items should not contain options that are
subsumed by other options.

• better
• A patient had a high fever, a greater blood
  pressure on the right side of his body than his
  left. He was loosing about one pound a day.
  Which of these caused this person to become ill?
  
• A. elephantiasis
• B. vasculitis
• C. gastritis
• pneumonia

52
Options should not be opposites of one another.

• poor
• What time will it be on the side of the earth
  facing the sun?
• Daytime
• Nighttime
• 200 A.M.
• 1107 P.M.

53

• better
• What causes there to be nighttime on Earth?
• A. One side of Earth is facing away from the Sun.
• B. The Earth revolves around to the back side of
  the Sun.
• C. The sun moves to the daylight side of Earth.
• D. The sun stops sending light toward Earth.

54
Wording and/or art must be clear.Which of
these models shows the orbit of the Earth?
55
Factual information should be accurate and
logical.
Onion root do not generally grow to a length of
300 centimeters and even if they did they would
not grow as fast as indicated by the graph.
56
Does the writer have a misconception?

• Five-fingers occur more commonly in any human
  population than six fingers. The five-finger
  trait is probably --
• A. dominant
• B. recessive
• C. hybrid
• D. incomplete

57
NOTE

• Every item can be improved.
• Better in this presentation does not equate
  to good.

58
Loricifera Use a Checklistto Explain How You
Arrive at Your Answers
59
Loriciferans

• Answering questions without knowing much about
  the subject

60

• 1. Which of the following is a characteristic
  unique to the phylum Loricifera?
• A. Multicellularity
• B. Small body structure
• C. A flexible feeding tube that can be retracted
  into the organism
• D. Multiple spines

61

• 2. All Loriciferans are
• Single cellular
• Multicellular
• Colorful
• Hungry

62

• 3. What is the level of classification for
  Loriciferans?
• Kingdom
• Phylum
• Genus
• Species

63

• 4. All of these are places Loriciferans can be
  found EXCEPT
• A. sediments
• B. artic ocean sands
• C. ocean coastal deposits
• D. in the needles of pine trees

64

• 5. Who named the phylum Loricifera?
• A. Reinhardt Kristensen, a Smithsonian
  Postdoctoral Fellow
• B. Carl Linnaeus, a Postdoctoral Fellow
• C. Feras Feras, a Lorci Postdoctoral Fellow
• D. A Postdoctoral Fellow

65

• 6. Loriciferans are most closely associated with
  an --
• a. glacier c. ocean
• b. river D. storm

66

• 7. What is the source of energy for
  Loriciferans?
• A. Snow
• B. Meiofauna
• C. Phyla
• D. Cars

67

• 8. Which of these do you think is the most
  important phylum?
• Phylum Arthropoda
• Phylum Loricifera
• Phylum Chordata
• Phylum Platyhelminthes

68

• 9. Loricifera larvae 1 and 2 are moving toward
  the grain of sand. If they start at the same
  time and arrive at the next grain of sand at the
  same moment, which one moved the fastest?
• A. 2 moved faster than 1
• B. 1 moved faster than 2
• C. 1 and 2 moved at the same speed
• D. 1 moved slower than 2

69

• 10. Which of these is not a characteristic of
  the producers Loriciferas eat?
• A. They are able to make their own food.
• B. They lack photosynthetic pigments.
• C. They do not require carbon dioxide at night.
• D. They give off oxygen when it is not dark.

70

• 10. What type of fauna is most appealing to
  Loriciferans?
• A. deer B. meiofauna
• C. fish D. flowers

71

• 11. Loriciferans are approximately 0.3 mm
  long. If 8 loriciferans were placed side by side
  they would extend approximately how many
  millimeters?
• A. 0.1 mm B. 2.4 mm
• C. 5.5 mm D. 1.6 mm

72

• 12. Which of the following pieces of equipment
  would be required to calculate the density of the
  water surrounding the sand in which the
  Loriciferans live?
• A. Balance and graduated cylinder
• B. Graduated cylinder and burner
• C. Thermometer and balance
• D. Balance and volumetric flask

73

• 13. Which of these is the main function of the
  ovaries of adult Loriciferans?
• A. Male
• B. Eggs
• C. Food
• D. Larva

74

• 14. What is the function of the protonephridium?
  
• A. To produce digestive enzymes
• B. To eliminate metabolic waste
• C. To pump blood
• D. To act as a primitive eye

75

• 15. What type of compounds have a multitude of
  different functions in Loriciferans including
  providing structural support within cells and
  catalyzing chemical reactions and facilitating
  diffusion through the plasma membrane and
  allowing cell recognition through its unique
  structure?
• A. mitochondria B. proteins
• C. ribosomes D. cells

76

• 16. In the Loriciferans ecosystem energy flows
  from producers to primary consumers to
  Loriciferans. More energy flows from producers
  to primary consumers in the Loriciferans
  ecosystem than in a forest ecosystem where there
  is thick vegetation. Why is this?
• A Forest plants receive more sunlight than
  aquatic photosynthetic organisms.
• B Plants in forests put more energy into wood and
  other hard-to-digest biomass.
• C Animals in oceans perform less cellular
  respiration than animals in forests.
• D Producers are less able to harvest the light
  energy in forests than in oceans.

77
Well Constructed Test Items

• Match the objective.
• Do not give away the answer.
• Do not favor the test wise student.
• Do not favor the good reader over the good social
  studies, math, or science student.

78
Well Constructed Test Items

• Avoid the use of negative words in the stem
  and/or options.
• Have responses in logical order.
• Have one clearly correct answer.
• Are factually correct.
• Make it clear what the student is being asked to
  do.

79
Well Constructed Options

• Parallel in degree of specificity.
• Parallel in length
• Parallel in grammatical structure.

80
Well Constructed Options

• Free of excess repetition
• Do not subsume other options
• Not opposite of one another
• Have rationales for the distractors

81
Did you Learn?

• Why test?
• What is being tested?
• What makes some test items better than others?
• There are no perfect items.

82

• The fatal pedagogical error
• to throw answers like stones at the heads of
  those who have not yet asked the questions.
• Paul Tillich

83
A Test A Day

• Keeps Students Away

84

• Teachers should not have to put aside good
  instruction to prepare for the test.
• Unknown

85
Every test question can be improved.
86
Every student can learn.
87
Every teacher can improve.
88
Every school can become a center of excellence.
89
No one said it will be easy.

• There are no short cuts.

90
The Road to Success Is Always Under
Construction.
91
For more information http//webusers.xula.edu/jse
venai/objective/objective.html Easy to follow
examples of test items http//www.tenet.edu/teks/
science/stacks/instruct/strategy.html Strengths
and weaknesses of various strategies. Provides
examples of poor and better selected response
items http//questioning.org/Q6/question.html
For more questions