Assessment and Instruction Aligned to Science Content Standards George E' DeBoer Deputy Director, AA

1
Assessment and Instruction Aligned to Science
Content StandardsGeorge E. DeBoerDeputy
Director, AAAS Project 2061

• Association of American Publishers
• Summit on Math and Science Education
• October 4, 2007
• Arlington, VA
• This work is funded by the National Science
  Foundation
• ESI 0352473

2
Background on Project 2061 (www.project2061.org)

• Project 2061 is a science education reform
  initiative of the American Association for the
  Advancement of Science (AAAS).
• Project 2061 began in 1985, two years after the
  1983 publication of A Nation at Risk.
• 2061 is the year the Halleys Comet will again be
  visible from earth, a metaphor for long-term
  reform.

3
2061 1985 1910 1835 1759 1682
4

• Science for All Americans (1989)
• A narrative account of ideas in science, social
  science, mathematics, and technology that all
  adults should know to be scientifically literate.

5

• Benchmarks for Science Literacy (1993)
• Learning goals to be met by the end of each of
  four grade bands (K-2, 3-5, 6-8, 9-12) to achieve
  the goal of science literacy for all.
• The first national science standards document.
  The emphasis is primarily on improved teaching
  and learning of science, not necessarily on
  accountability.

6
Making Connections Between Science Ideas
Strand maps show the connections among science
ideas. Used to facilitate coherence in
instruction and assessment (2001,2007). Articulat
ion from K-12
7
(No Transcript)
8

9
Two main thrusts of our current work

• The development of tools and resources to support
  effective goals-based instruction
• The development of tools and resources to support
  effective goals-based assessment

10
We begin by writing clarification statements to
specify what students should know and be able to
do. (Standards by themselves tend to be
under-specified.)

• In 2001, the Commission on Instructionally
  Supportive Assessment said A states high
  priority content standards must be clearly and
  thoroughly described so that the knowledge and
  skills students need to demonstrate competence
  are evident. This should result in relatively
  brief, educator-friendly descriptions...

11
Benchmarks, key ideas, and clarifications

• Atoms and molecules are perpetually in motion.
  Increased temperature means greater average
  energy of motion, so most substances expand when
  heated. (Benchmark 4D/M3ab)

12
Key Idea 1 Atoms and molecules are perpetually
in motion.

• Students should know that atoms and molecules of
  all matter are always moving. They are expected
  to know that this is true for atoms or molecules
  of solids, liquids, and gases.  They are expected
  to know that, even when objects that are made up
  of these atoms and molecules appear not to be
  moving, the atoms and molecules that make up
  those objects are nonetheless themselves in
  constant motion. Students should know that the
  motion of atoms or molecules can include moving
  back and forth with respect to a fixed point,
  around a fixed point, and/or past each other from
  one fixed point to another

13
Atoms/Molecules of Solids and Liquids in Motion

• There is a solid wooden table with a cup of water
  sitting on it. Which of the following statements
  about the atoms and molecules of the table and
  the atoms and molecules of the water is TRUE?
• A. The atoms and molecules of both the liquid
  water and the table are moving.
• B. The atoms and molecules of both the liquid
  water and the table are not moving.
• C. The atoms and molecules of the liquid water
  are not moving, and the atoms and molecules of
  the table are moving.
• D. The atoms and molecules of the liquid water
  are moving, and the atoms and molecules of the
  table are not moving.

14
Atoms/Molecules of Solids and Gases in Motion

• A balloon full of air is placed on a chair. Which
  of the following statements about the atoms and
  molecules of the chair and the atoms and
  molecules of the air in the balloon is TRUE?
• A. The atoms and molecules of both the chair and
  the air in the balloon are moving.
• B. The atoms and molecules of both the chair and
  the air in the balloon are not moving.
• C. The atoms and molecules of the chair are not
  moving, and the atoms and molecules of the air in
  the balloon are moving.
• D. The atoms and molecules of the chair are
  moving, and the atoms and molecules of the air in
  the balloon are not moving.

15
Results

• Molecules of gases in motion 79.3
• Molecules of liquids in motion 69.7
• Molecules of solids in motion 43.5

16
Key Idea Increased temperature means greater
average energy of motion so most substances
expand when heated.

• Students should know that as the temperature of a
  substance increases the distance between the
  atoms and/or molecules of the substance typically
  increases and the substance expands. Students
  should know the reverse of this expansion occurs
  when the temperature of a substance is decreased.
  They should know that as the temperature
  decreases, the distance between the atoms and/or
  molecules decreases and the substance contracts.
  Students are expected to know that this expansion
  or contraction can happen to solids, liquids, and
  gases

17
Sample Assessment ItemIron Frying Pan Item
(Atoms-only version)

• After cooking breakfast, a cook places a hot iron
  frying pan on the counter to cool.  What happens
  as the iron pan cools?
• The iron atoms get heavier.
• The iron atoms decrease in size.
• The number of iron atoms increases.
• The distance between iron atoms decreases.

18
Iron Frying Pan Item (Atoms Macro-phenomenon)

• After cooking breakfast, a cook places a hot iron
  frying pan on the counter to cool.  What happens
  as the iron pan cools?
• Even though you cannot see it, the pan gets a
  tiny bit smaller because the iron atoms decrease
  in size.
• Even though you cannot see it, the pan gets a
  tiny bit smaller because the distance between
  iron atoms decreases.
• Even though you cannot feel it, the pan gets a
  tiny bit heavier because the iron atoms increase
  in mass.
• Even though you cannot feel it, the pan gets a
  tiny bit heavier because the number of iron atoms
  increases.

19
Results of Pilot Testing
20
Sample Phenomenon to Illustrate Key Idea Thermal
Expansion of a Solid

• Students observe that a metal ball that fits
  through a metal ring will no longer fit through
  the ring after the ball is heated.
• Students need to interpret the lack of fitting
  as an indication of the thermal expansion of the
  metal ball.
• To help students reconcile this phenomenon with
  their everyday observations that macroscopic
  substances dont appear to expand or contract,
  students need to appreciate that the
  ball-and-ring device is capable of detecting
  small changes that their eyes may not detect.

http//www.sciencekit.com/category.asp_Q_c_E_42962
8
21
Sample Phenomenon to Illustrate Key Idea Thermal
Expansion of a Liquid

• Students observe that the level of liquid
  mercury rises as a thermometer is heated.
• Students need to interpret the height increase of
  the liquid mercury as an indication of its
  thermal expansion.
• To help students reconcile this phenomenon with
  their everyday observations that macroscopic
  substances dont appear to expand, students need
  to appreciate that the tiny diameter of the
  thermometer makes it easier to detect the change.

http//sol.sci.uop.edu/jfalward/temperatureandexp
ansion/temperatureandexpansion.html
22
Findings related to food for growth

• Many students think that food provides energy for
  growth but not materials for growth.
• Item For an animal to grow, what must happen to
  the food that it eats?
• The food must be changed into energy or
  eliminated as waste. 42.6
• The food must be broken down into simpler
  substances that become part of the animals
  body. 14.8
• Middle school students typically dont think
  about where the mass for growth comes from. They
  dont worry about conservation of mass.
• Current instruction focuses almost entirely on
  the energy story almost nothing on the matter
  story. Instruction is needed to counter the idea
  that things just grow.

23
Findings related to food web diagrams

• On items using food web diagrams, students have
  difficulty with indirect effects and with items
  that use symbols instead of names of organisms.

24
Assessing Student Understanding of Direct
Effects

•                             WORMS ?  ROBINS ?
  FOXES           
• If a disease kills most of the worms, which
  of the following statements describes what will
  happen to the robins?
• 75 correct

25
Assessing Student Understanding of Indirect
Effects

•                           WORMS ?    ROBINS ?
  FOXES                       
• If a disease kills most of the worms, which
  of the following statements describes what will
  happen to the foxes?
• 45 correct

26
Assessing Student Understanding with Symbols for
Organisms

•                  A ?   B ? C
                                 
• If a disease kills most of the As, which
  of the following statements describes what will
  happen to population B?
• 36 correct

27
The Current Work on Assessment

• We are creating a bank of middle school science
  assessment items that are precisely aligned with
  national content standards and that can be used
  for diagnostic purposes.
• A resource for educators

28
Item Development

• Two-year item development cycle
• Clarify learning goals.
• Review the research literature on student
  learning.
• Design items that are content-aligned and that
  use misconceptions as distractors.
• Pilot test each item (about 150 students per
  item). Students are asked to give us feedback
  about the item.
• Teams of external reviewers formally evaluate
  items.
• Field test each item (about 1000 students per
  itemnational sample).

29
Assuring Content Alignment Criteria of Necessity
and Sufficiency

• Necessity Knowledge of the idea described in the
  learning goal must be needed to evaluate each of
  the answer choices.
• Sufficiency Knowledge of the idea described in
  the learning goal should be enough by itself to
  successfully evaluate each of the answer choices.

30
Assuring Construct Validity

• Items should be comprehensible and accessible to
  all students (use plain language, i.e., simple
  vocabulary and sentence structure)
• Task contexts should not advantage or
  disadvantage one group of students because of
  their interest or familiarity with the context.
• Items should not be easy to answer by using
  test-wiseness strategies.

31
The Current Work in Instruction

• We are developing a data base of phenomena and
  representations linked to content standards, and
  closely linked to our work in assessment.
• A resource for educators

32
Some Thoughts about How to Improve Science
Education
33
What we can do to improve science education

• Focus on the most important and teachable ideas
  in science. We cant teach everything.
  Standards point the way.
• Use assessment for diagnostic purposes.
  Assessment is not just about accountability.
• Connect science ideas to real-world phenomena.
  Phenomena alone and abstract principles alone are
  not effective in achieving the goal of science
  literacy.
• Embed pedagogical support into curriculum
  materials (e.g., to address the key
  misconceptions students have, to provide
  opportunities for students to make sense of
  science ideas, and to make the connections among
  ideas clear).

34
One More Thought

• Can researchers and implementers work more
  closely together? Project 2061, for example, is
  a research group without an implementation arm.
  I assume there are publishers without a research
  arm. Is more collaboration possible?