A Standard Survey for Undergraduate Non-Science Majors: The Astronomy Diagnostic Test

1
A Standard Survey for Undergraduate
Non-Science Majors The Astronomy Diagnostic
Test

• Beth Hufnagel
• University of Maryland, USA
• hufnagel_at_astro.umd.edu
• 7 August 1999 AAPT Summer MeetingSan Antonio,
  Texas

2

• Faculty E. F. (Joe) Redish David Hammer John
  Layman (emeritus)
• Grad Students Aziza Baccouche Lei Bao Hadley
  Lawler Rebecca Lippmann Mel Sabella

• Postdocs Richard Steinberg Michael
  Wittmann Andy Elby Apriel Hodari Beth
  Hufnagel ( NAE Spencer Fellow PFSMETE
  Fellow)

3
OverviewAstronomy Education Research
Students knowledge andbeliefs about
The UniverseConceptions
KnowingEpistemology
ThemselvesAffect
Development and assessment of curricula and
teaching techniques
4
Measuring Concept Learning Why A Standardized
Survey?

• Understand student preparedness.

• Guide selection of curriculum/teaching approach.

• Measure the effectiveness of different teaching
  approaches on learning.

• Document the effectiveness of teaching.

5
Collaboration for Astronomy Education Research
(CAER)
Montana State Jeff Adams Christine Brick Tim
Slater
Univ. of Maryland Grace Deming Beth Hufnagel
Univ. of New Mexico Michael Zeilik
Univ. of Nebraska Rebecca Lindell Adrian
6
What is theAstronomy Diagnostic Test?

• 33 multiple-choice questions
• 21 concept/knowledge questions
• 12 student background questions

• Validated and Reliable
• A wrong answer means the student doesnt
  understand.
• A right answer usually means the student
  understands.

• Diverse Comparison Database
• 22 classes ranging from 6-201 students
• 17 institutions of 5 types

7
Pre-Course Results Spring 1999Sample ADT
Question8. Where does the Suns energy come
from?
No. 420 27 870 56 142 9 122
8 3 0

• A. The combining of light elements into heavier
  elements.
• B. The breaking apart of heavy elements into
  lighter ones.
• C. The glow from molten rocks.
• D. Heat left over from the Big Bang.
• Did not attempt to answer.
• 
  Total

1557 100
8
1998/1999 Development of ADT version 2.0

• The choice of ADT topics was based on two prior
  diagnostics by Phil Sadler and Mike Zeilik.

• The ADT version 1.0 was re-written using standard
  psychometric rules.

• 30 student written responses to open-ended
  questions.

• 2500 students in 26 institutions of 6 types
• Statistics, e.g., Spring 1999Discrimination
  range 0.320.67 (out of 1.00 to 1.00).

• Validation based on 60 interviews,
• open-ended in Fall 1998,
• with distractors in Spring 1999.

9
Do the results depend on
Research Questions

• the type of institution - community college,
  liberal arts, state university, or
  technical? Yes

• the class size?
  No
  

• gender?
  Yes Why?
  

10
Results from Spring 1999Pre-Course Scores by
Gender
Female Male N 825 683
Mean 28 38 Std. Error 0.4 0.6

• Gender matters.

11
Summary

• The ADT is now available in PDF format from
  http//solar.physics.montana.edu/aae/adt/.

• Encourage conceptual understanding as a goal for
  astronomy courses.

• More quantitative measurement of the
  effectiveness of curricula/learning tools.

• CAER will develop a web-based, ADT database with
  independent concept clusters.

12
Favorite Quote from Student Interview

• I seem to know, or think I know, a lot of
  things. I just dont know the insides of
  them.
• - Debbie(Undergradua
  te astronomy student during the last week of the
  course.)

13
Thanks to these generous participants
14

• Phil Appleton, Iowa State University
• Chris Depree, Agnes Scott College
• Van Dixon, Vista Community College
• Andrea Dobson, Whitman College
• Jack Dostal, Hawkeye Community College
• Doug Hamilton, University of Maryland at College
  Park
• Mary Kay Hemenway, University of Texas at Austin
• Thomas J. Kassebaum, Denison University
• Janet Landato, W.R. Harper College
• Bernie McNamara, New Mexico State University
• Heather Morrison, Case Western Reserve University
• Jeff Saul, North Carolina State University
• Steve Shawl, University of Kansas
• Harry Shipman, University of Delaware
• Patrick Shopbell, Prince Georges Community
  College
• Greg Stacy, Southern University
• Bill Warren and Rolfe Chandler, Lord Fairfax
  Community College

15
Results from Spring 1999 Pre-Course Scores by
Class Size

• The average pre-course score does not depend on
  class size.

16
Results Spring 1999 Pre-course Scores by
Institution/Gender
Classes Students
Women Men State Universities 11 635/499
28(1.0) 37(1.5) Liberal Arts 5
110/124 26(1.1) 38(5.1) Community Coll. 4
52/43 28(1.7) 39(2.6) Technical
1 6/17 29(14) 59(21) Womens
1 22/0 30(8)
n/a (standard error in
parentheses)

• In all 22 classes, the average pre-course score
  for men is higher than the average score for
  women.

17
Implications for Instruction Explicitly Evoke
Useful Ideas

• Students come to an introductory astronomy class
  with a great deal of knowledge. A professor can
  help her students use their existing correct and
  relevant ideas to construct theories closer to
  those held by astronomers.
• For example, all of the students knew that the
  Sun is hot, and that fluids (like coffee) will
  cool if no heat source is present. Use a
  demonstration to encourage them to apply this
  knowledge to stars so that they will conclude
  stars need an internal heat source.

18
San Antonio AAPT Summer Meeting

• Please Add Name for Preprints