Increasing Reading Comprehension Achievement through Item Analysis

1
Increasing Reading Comprehension Achievement
through Item Analysis

• Monday, February 11, 2008
• Auditorium III
• 215 315 p.m.

2
Reading Achievement

• State end-of-grade reading comprehension scores
  have been flat in recent years.
• NAEP reading comprehension scores have been
  declining in recent years.

3
Reading Achievement

• In 2005, North Carolina 8th graders fell below
  the national average on NAEP.
• The State Board of Education proposed literacy
  coaches at the middle grades in response to this
  concern.

4
(No Transcript)
5
(No Transcript)
6
Levels of Thinking and Reasoning (Adapted from
Marzano, et al, 1988)

• Knowledge
• Organizing
• Applying
• ______________
• Analyzing
• Generating
• Integrating
• Evaluating

• Usually lower order thinking skills
• Usually higher order thinking skills

7
Knowledge

• Involves recall of a fact
• Example Where do the insects live?

8
Organizing

• Involves the order of information, comparing and
  contrasting, or classifying information
• Example How does Joes job differ from Jims
  job?

9
Applying

• Involves demonstrating prior knowledge within a
  new situation
• Example Which statement from the selection is
  true?

10
Analyzing

• Involves clarifying information, determining
  characteristics or parts, identifying
  relationships and problems, and identifying main
  ideas
• Example Which words help create the somber mood?

11
Generating

• Involves producing new information, inferring,
  predicting, or explaining something by adding
  examples
• Example How would this experience help Sally the
  next time she goes camping?

12
Integrating

• Involves connecting and combining information and
  summarizing
• Example Which experience is most similar to
  Tims experience in the selection?

13
Evaluating

• Involves judging the quality of ideas,
  determining if ideas are valid, establishing
  criteria, etc.
• Example Why did the author begin the selection
  with an untrue statement?

14
Thinking Skills Activity

• Read the passage Ellis Island Doorway to
  America.
• Work in small groups to analyze the questions and
  determine the appropriate thinking skill for each
  question.

15
Item Results

• Thinking Skill Analyzing (13)
• North Carolina 89
• National Public 91
• Below Basic 210

16
Item Results

• Thinking Skill Knowledge
• North Carolina 87
• National Public 89
• Below Basic 228

17
Item Results

• Thinking Skill Applying
• North Carolina 81
• National Public 82
• Below Basic 237

18
Item Results

• Thinking Skill Analyzing (11)
• North Carolina 72
• National Public 75
• Basic 253

19
Item Results

• Thinking Skill Generating (15)
• North Carolina 54
• National Public 59
• Basic 267

20
Item Results

• Thinking Skill Generating (16)
• North Carolina 32
• National Public 39
• Proficient 318

21
Strategies

• Ask higher order thinking questions in
  instruction.
• Utilize the higher order questions in textbook
  supplementary materials.
• Provide classroom assessments with higher order
  questions.
• Open-ended as well as multiple-choice

22
Strategies

• Encourage students to ask higher order questions
  of text.
• Have students write questions for each other.
• Independent reading and journaling

23
Strategies

• Teachers collaborate with peers to generate
  higher order questions for instruction and
  assessment
• Assists with internalizing thinking skills
• Assists with identifying the higher order aspects
  of the text

24
Strategies

• Change lower order questions into higher
  questions by rewriting the question.
• Examples

25
Improving Lower Order Questions

• Lower Order Question
• What happened to Johnny when he bought the
  bicycle?
• Knowledge because it is stated in the selection.
• Higher Order Question
• How will this experience most likely change what
  Johnny does the next time he buys a bicycle?
• Generating because it involves making a
  prediction about what will happen next.

26
Improving Lower Order Questions

• Lower Order Question
• What is the personification in paragraph 2?
• Knowledge question
• Higher Order Question
• What is the effect of the use of personification
  in paragraph 2?
• Analyzing question because it requires the
  student to not only recognize personification,
  but also to analyze the effect it has on the
  paragraph.

27
Resources

• NAEP Data Explorer http//nces.ed.gov/nationsrep
  ortcard/nde
• NAEP Questions Tool http//nces.ed.gov/nationsre
  portcard/itmrls
• NAEP Item Maps http//nces.ed.gov/nationsreportc
  ard/itemmaps
• NC EOG Sample Items http//www.ncpublicschools.o
  rg/accountability/testing/eog/sampleitems/reading
• Marzano, R.J., Brandt, R.S, Hughes, C.S., Jones,
  B.F., Presseinsen, B.Z., Stuart, C., Suhor, C.
  (1988). Dimensions of Thinking. Alexandria, VA
  Association for Supervision and Curriculum
  Development.

28
Questions
29
Increasing Reading Comprehension Achievement
through Item Analysis

• Tammy Howard
• ELA Test Development Consultant
• Marcie Hickman
• NAEP State Coordinator