Measuring Complex Achievement: Essay Questions

1
Measuring Complex Achievement Essay Questions

• Chapter 10
• Jessica Rigsby

2
Objectives for Chapter 10

• Discuss the use of essay questions
• Compare two types of essay questions
• Hypothesize the advantages and disadvantages of
  using essay questions
• Compare two types of scoring rubrics
• Evaluate sample essays using a scoring rubric
• Create examples of two types of essay questions

3
Whats So Great About Essay Questions?

• They allow assessment of students abilities to
• Formulate problems
• Organize, integrate, and evaluate ideas and
  information
• Apply knowledge and skills

4
Forms and Uses of Essay Questions

• May be used to measure knowledge of factual
  information
• Full potential realized with more higher-order
  thinking problems

5

• Useful in meeting learning outcomes concerned
  with the abilities to conceptualize, construct,
  organize, integrate, relate, and evaluate ideas
  in content areas such as history, civics,
  literature, science, math, etc.
• Most widely used when the main focus is on
  student writing in any subject
• Examples distinguishing between narrative
  essays, expository essays, and persuasive essays
  or focusing on writing process

6

• Confucius Say
• The freedom of response provided by essay
  questions is not an all-or-nothing affair but,
  rather, a matter of degrees.

Restricted Response
Extended- Response
7
Restricted Response Essay Questions

• Usually limits both the content and the response
  by restricting the scope of the topic to be
  discussed, generally indicated in the question
• Useful for measuring learning outcomes requiring
  interpretation and application of data in a
  specific area
• Any outcomes measured by an objective
  interpretive exercise can be measured by a
  restricted response essay question

8
Examples of Restricted Response Essay Questions

• Describe two situations that demonstrate the
  application of the law of supply and demand. Do
  not use those examples discussed in class.
• State the main differences between the Vietnam
  War and previous wars in which the United States
  has participated.
• Why is the barometer one of the most useful
  instruments for forecasting weather? Answer in a
  brief paragraph.

9

• The Restricted Response essay question provides
  for more ease of assessment, but it restricts the
  scope of the topic to be discussed and indicates
  the nature of the desired response to the student
  which limits his or her opportunity to
  demonstrate these behaviors on their own.
• Is there a better method for evaluating complex
  achievement

10
Extended-Response Essay Questions

• Freedom of response allows student to select
  information that they think is pertinent, to
  organize the answer in accordance with their best
  judgment, and to integrate and evaluate ideas as
  they deem appropriate
• Places value on higher-order thinking skills

11
Examples of Extended-Response Essay Questions

• Imagine that you and a friend found a magic wand.
  Write a story about an adventure that you and
  your friend had with the magic wand.
• Compare developments in international relations
  in the administrations of President William
  Clinton and President George W. Bush. Cite
  examples when possible.

12
To Keep Things Simple

• Objective Interpretive- select
• Restricted Response Essay- supply
• Extended-Response Essay- write

13
Pros Cons of Essay Questions

• Apply your knowledge
• They give student the options to chose what they
  want to write about
• reasoning
• The teacher can tell if the student studies or
  not
• Can often prepare in advance what to write

• Time consuming
• Grading grammar usage vs. content
• Legibility
• Hard to formulate ideas in time
• Takes longer to grade

14
How Can We Change Those Cons to Pros?

• Unreliability- clearly defining learning outcomes
  to be measured, properly framing questions,
  carefully following scoring rules, and obtaining
  practice in scoring
• Amount of time- reserve use of extended-response
  questions for learning outcomes that cannot be
  measured well objectively

15

• Limited Sampling- try to obtain as representative
  a sample of learning outcomes as possible
  (accumulate a series of essays throughout the
  school year to be included in a writing
  portfolio)

16
Suggestions for Constructing Essay Questions

• Restrict use of essays to learning outcomes that
  cannot be measured well objectively
• Construct questions that call forth skills
  specified in learning standards
• Example box on page 235

17

• Phrase the question so that the students task is
  clearly indicated
• Example
• Poor Compare the Democratic and Republican
  parties.

18

• Better Compare the current policies of the
  Democratic and Republican parties with regard to
  the role of government in private business.
  Support your statements with examples when
  possible. (Your answer should be confined to two
  pages. It will be evaluated in terms of the
  appropriateness of the facts and examples
  presented and the skill with which it is
  organized.)

19

• Indicate an approximate time limit for each
  question
• Avoid the use of optional questions
• Example Box page 239
• Now that we have essay questions how do we score
  them?

20
Scoring Essay Questions

• Tips to remember
• Use clear specifications of scoring criteria
• Inform students of scoring criteria
• Use an initial review to find anchor responses
  for comparison
• Use descriptive rather than judgmental scores or
  levels (writing is clear and thoughts are
  complete vs. excellent)

21
Scoring for Restricted Response Essay Questions

• In most instances, the teacher should write an
  example of an expected response
• For example, if the student is asked to describe
  three factors that contributed to the start of
  the Civil War, the teacher would construct a list
  of acceptable reasons and give the student 1
  point for each of up to three reasons given from
  the list

22
Scoring for Extended-Response Essay Questions

• Analytic Scoring Rubrics
• Consist of a rubric broken down into key
  dimensions that will be evaluated
• Enables teacher to focus on one characteristic of
  a response at a time
• Provides maximum feedback for students

23

• Holistic Scoring Rubrics
• Yield a single overall score taking into account
  the entire response
• Can be used to grade essays more quickly
• Does not provide as much specific feedback as
  analytic rubric
• Should not consist of scores alone, but rather
  contain scores accompanied by statements of the
  characteristics of the response
• Example Table 10.3 and 10.4

24
Suggestions for Scoring Essay Questions

• Prepare an outline of the expected answer in
  advance and use a clear scoring rubric
• Use the scoring rubric that is most appropriate
• Decide how to handle factors that are irrelevant
  to the learning outcomes being measured

25

• Evaluate all responses to one question before
  going on to the next one
• When possible, evaluate answers without looking
  at the students name
• If especially important decisions are to be based
  on the results, obtain two or more independent
  ratings
• Look out for bluffing! Page 247