Best Practices for Building Tests and Quizzes - PowerPoint PPT Presentation

Loading...

PPT – Best Practices for Building Tests and Quizzes PowerPoint presentation | free to download - id: 22f345-ZDMzY



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Best Practices for Building Tests and Quizzes

Description:

Essays measure the student's ability to communicate effectively, not just their ... http://www.glencoe.com/sec/teachingtoday/educationupclose.phtml/print/40 ... – PowerPoint PPT presentation

Number of Views:71
Avg rating:3.0/5.0
Slides: 42
Provided by: Anthony338
Learn more at: http://anthonyfoster.com
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Best Practices for Building Tests and Quizzes


1
Best Practices for Building Tests and Quizzes
  • League for Innovation
  • November 2004

2
7 question types
  • multiple choice
  • true/false
  • essay/short answer
  • matching
  • ordering
  • fill in the blank
  • multiple answers

3
Part II-Instructional Design Considerations for
Creating Tests
4
Selected Response Questions
  • Selected response questions require students to
    select from a predetermined list of potential
    answers.
  • multiple choice
  • true false
  • Matching
  • fill-in-the blank questions

5
Created Response Questions
  • Ex. Essays measure the students ability to
    communicate effectively, not just their
    understanding of content.
  • Easier to write but harder and (more subjective)
    to grade.

6
Match Question Type to Level of Assessment Desired
  • Multiple choice and matching questions
  • offer the most flexibility in terms of content
    covered
  • thinking skills that can be assessed
  • True/false are usually limited to fact recall.

7
Match Instructional Objectives
  • Use lesson plans or teacher notes to accurately
    reflect content that was covered in class
  • Choose the most important objectives to assess
    and use these as the outline for your test

8
Cover Important Material
  • Facts, definitions, comprehension, analysis,
    applications
  • Trivial items result in trivial studying and
    learning.

9
Items Should be Independent
  • Do not give away answers via information in
    other questions
  • Independence maximizes breadth of coverage

10
Write Simply and Clearly
  • Measure knowledge of material and concepts, not
    vocabulary
  • Ambiguous questions create error, frustration,
    and compound biases related to language and
    disability

11
Clearly Specify What Type of Response is Sought
  • How long or short an answer is sought?
  • Should they show their work?
  • Do you want description? comparisons?
    application? evaluation?
  • Whose opinion do you want (book, lecture, their
    own)?

12
Good Tests Take Time to Write
  • Give yourself enough time to evaluate items after
    a day or two
  • Revise, edit, and ask others to read before
    administering the test

13
Good Tests Have High Degrees of Reliability and
Validity
  • Reliability refers to the extent to which
    measurement is consistent
  • Validity refers to the extent to which a test
    measures what it should.

14
Poor Example
15
Construct Questions Carefully
  • The most important test-making process is
    selecting the wording of each question and
    answer.
  • Question Stem and Distractors

16
Construction Tips
  • Construct evaluation items with a single correct
    answer
  • Use plausible distracters
  • Arrange options in a logical sequence
  • Include a minimum of 3 and a maximum of 5 options
  • Alternate the order of the correct responses
    randomly among all options

17
Construction Tips
  • Refrain from using the choices all of the above
    or none of the above
  • Identify an initial difficulty level

18
Construction Tips
  • Increasing item difficulty can be achieved by
    paying attention to the distracters (options)
  • The more homogeneous the options and
    distracters, the higher the degree of difficulty

19
Higher Degree of Difficulty
  • The first James Bond movie was released in what
    year?
  • a) 1960
  • b) 1962
  • c) 1964
  • d) 1966

20
Mid-Level Difficulty
  • The first James Bond movie was released in what
    year?
  • a) 1960
  • b) 1962
  • c) 1964
  • d) 1970

21
Low-Level Difficulty
  • The first James Bond movie was released in what
    year?
  • a) 1958
  • b) 1962
  • c) 1970
  • d) 1975

22
Points of Concern for Different Formats
  • Writing True-False Items
  • Choose words with precise, definite meanings
  • Avoid tricks and trivia
  • Avoid easy clues
  • Follow guidelines for multiple choice

23
Writing Multiple Choice Questions
  • Present the problem, including qualifying
    statement
  • There should be only one correct answer
  • Distracters should be plausible but clearly
    incorrect
  • Avoid negative wording (especially double
    negatives)

24
Writing Multiple Choice Questions
  • When item is controversial, indicate whose
    opinion is sought
  • Avoid irrelevant cues to correct answer (length,
    grammar)
  • Items should test one central idea or concept

25
Writing Multiple Choice Questions
  • Watch out for patterns in alternatives (overuse
    of "C" as correct answer)
  • Choose appropriate level of difficulty
  • Assure even coverage of material and types of
    knowledge

26
Writing Matching Items
  • Keep each matching set short (no more than 5
    stimuli/responses)
  • Use longer items as stimuli, shorter as responses
  • Arrange responses in alphabetical or logical
    order

27
Writing Short Answer and Completion Items
  • Clearly indicate type of answer you want
  • Do not use more than two blanks per completion
    item
  • Make a key before scoring
  • Periodically re-score early tests to detect
    shifting criteria

28
Writing Essay Questions
  • Limit questions to vitally important material
  • Clearly define task, scope, and directions for a
    "good" answer
  • Allow time for thought
  • Use multiple medium-length essays rather than one
    long one

29
Writing Essay Questions
  • Use questions that have a limited number of good
    answers
  • Allow choice between alternatives (e.g., "answer
    3 of the 4 questions")

30
Part III- Evaluating your Assessment or Assessing
Your Evaluation
31
Evaluating the Test
  • Once scored, spend some time reviewing the test
    and observing patterns that may be present.
  • Were there any questions that every student got
    wrong?
  • If so, can you deduce whether it was due to poor
    test item construction or to instruction?

32
Evaluating the Test
  • If poorly written, you may want to consider
    canceling it out and recalibrating the test.
  • If students didnt understand the concept,
    perhaps the content needs to be retaught.

33
A Checklist for Teachers
  • Are your directions CLEAR and CONCISE?
  • Are your objectives clear?
  • Is there a logical connection between questions
    and answers?
  • Are you being specific in regard to the area you
    are covering?
  • Are your answers simple and factual?

34
Higher Order Thinking
  • What later events best affirm the ideas set forth
    in the Declaration of Independence?
  • A) Emancipation Proclamation 19th Amendment
  • B) Eminent Domain, Manifest Destiny
  • C) Civil War

35
References
  • Orlich,Donald C, Harder,Robert J.,Callahan,Richard
    C, and Gibson,Harry W.(1998), Teaching
    Strategies A guide to better instruction, 5th
    Ed, Houghton Mifflin, New York
  • Royse, David (2001) Teaching Tips for College and
    University Instructors A Practical Guide, Allyn
    and Bacon, Boston

36
References
  • Dominowski, Roger L. (2002),Teaching
    Undergraduates, Lawrence Erlbaum Associates, New
    Jersey
  • Paul Eggen, Don Kauchak (2001),Educational
    Psychology, Windows on Classrooms, 5th Ed, Pub.
    Merrill- Prentice Hall, Upper Saddle River, New
    Jersey.

37

References
  • Bloom, B.S. (Ed.) (1956) Taxonomy of educational
    objectives The classification of educational
    goals Handbook I, cognitive domain. New York
    Toronto Longmans, Green
  • Chase, C.I. (1999).  Contemporary Assessment for
    Educators.  New York Addison Wesley Longman,
    Inc. 
  •  

38

References
  • Dominowski, R.L. (2002).  Teaching
    Undergraduates.  Mahwah, NJ  Lawrence Erlbaum
    Associates. 
  • Friedenberg, L. 1995. Psychological testing
    Design, analysis, and use.  Needham Heights
    Allyn Bacon.  
  •  

39
References
  • Kay Burke (1999), The Mindful School How to
    Assess authentic Learning Pub.SkyLight
    Professional Development , Arlington Heights,
    Illinois
  • Heinich, R., Molenda, M., Russell, J. D., and
    Smaldino, S. E. (1996). Instructional media and
    technologies for learning (5th ed.). (pp. 40-41).
    Englewood Cliffs Prentice-Hall.

40
References
  • Jacobs, L. C. Chase, C. I.  (1992). Developing
    and using tests effectively  A guide for
    faculty.  New York  Jossey-Bass.
  • Kopeikin, Hal S. (n. d.) UC Los Angeles
    lecturer, unpublished notes on writing effective
    questions.
  • http//web.wi.mit.edu/sive/pub/Lab20Resources20D
    ocuments/Thoughts20on20Exam20Quests.html

41
References
  • http//www.rpi.edu/verwyc/chap2tm.htm
  • http//www.glencoe.com/sec/teachingtoday/education
    upclose.phtml/print/40
  • http//taesig.8m.com/createv.html
  • http//www.utc.edu/Administration/WalkerTeachingRe
    sourceCenter/FacultyDevelopment/Assessment/test-qu
    estions.html
  • http//www.psych.ucsb.edu/taap/writingtests.html
    (defunct)

42
Writing Exam Questions and Constructing a Test
  • Anthony Foster
  • Senior Instructional Designer

43
The Task at Hand
  • Creating effective tests is an essential task for
    all classroom teachers
  • While Course management tools allow for ease of
    distributing quizzes and tests online, they do
    nothing to help with sound practices.
  • Lets examine the importance of good test
    question construction and
  • Provide a variety of solid suggestions you can
    follow when constructing your next classroom test

44
The Importance of Good Tests
  • A well-written test allows you to accurately and
    consistently measure students mastery of
    specific content taught in class
  • Results of well-written tests also allow teachers
    to measure, to some degree, how effective their
    instruction has been

45
The Danger of Bad Tests
  • Poorly designed test questions can lead to
  • inaccurate measurements of learning and provide
    false information regarding student performance
  • decreased effectiveness of instruction
  • unintended measurements of skills not taught

46
I. Guidelines for Good Tests
47
Cover important material
  • Facts
  • Definitions
  • Comprehension
  • Analysis
  • Applications
  • Trivial items result in trivial studying and
    learning

48
Strive for proportional representation
  • Make sure lectures, readings, and the key
    elements of the course are measured without too
    much emphasis on any one area

49
Items should be independent
  • Do not give away answers via information in other
    questions
  • Independence maximizes breadth of coverage

50
Write simply and clearly
  • Measure their knowledge of material, not
    vocabulary or "mind reading"
  • Tricky or ambiguous questions create error,
    frustration, and compound biases related to
    language and disability

51
Clearly specify what type of response is sought
  • How long or short an answer is sought?
  • Should they show their work?
  • Do you want
  • description?
  • comparisons?
  • application?
  • evaluation?
  • Who's opinion do you want (book, lecture, their
    own)?

52
Good items and therefore good tests take time to
write
  • Give yourself enough time to evaluate items after
    a day or two
  • Revise, edit, and ask others to read before
    administering

53
Good tests have high degrees of reliability and
validity
  • Reliability refers to the extent to which
    measurement is consistent
  • Validity refers to the extent to which a test
    measures what it should

54
Types of Test Questions
  • The test questions most teachers use in
    classrooms fall into two broad categories
  • selected response and
  • created response

55
Selected response questions
  • Require students to select from a predetermined
    list of potential answers
  • Multiple choice
  • True/false
  • Matching
  • Fill-in-the-blank questions

56
Selected response questions
  • Often viewed as less challenging in terms of the
    thinking skills required to answer them
  • When well written, they can measure higher levels
    of thinking, not simply the recalling of facts
  • The writing of these test items can be
    challenging, and they frequently take more time
    to construct
  • When well written they are easier to score and
    can provide a more objective method of assessment
    than do created response items

57
Created response questions
  • include extended answer and essays
  • measure the students ability to communicate
    effectively, not just their understanding of
    content

58
Created response questions
  • Several considerations when choosing what to
    include in a test
  • fair assessment of knowledge or skills,
  • number of students taking the test
  • amount of time available to score the test
  • This type is often easier to write, but can
    require more time to score
  • Scoring often less reliable, more subjective

59
Writing Better Tests
  • There are a number of steps you can take to
    improve the quality of the tests you write

60
Match Instructional Objectives and Teacher Notes
to the Test
  • When creating your test items...
  • Refer back to your instructional objectives for
    the content and skills you want to assess
  • Use any lesson plans or teacher notes regarding
    what was covered and assigned to ensure your test
    items accurately reflect content that was covered
    in class.
  • Choose the most important objectives to assess
    and use these as the outline for your test.

61
Match Question Type to Level of Assessment Desired
  • Carefully choose the type of questions
  • Multiple choice and matching questions offer the
    most flexibility in terms of content that can be
    covered and thinking skills that can be assessed
  • T/F are usually limited to fact recall
  • Try to balance the number of question types and
    limit their number to no more than three types on
    one test for middle school and high school
    students

62
Construct Questions Carefully
  • The next most important test-making process is
    selecting the wording of each question and answer
  • The stem or question portion of the test item
    should be carefully worded using straightforward
    language
  • Careful attention should also be given to writing
    the correct and incorrect answers (also known as
    distractors)

63
II. Advantages and Disadvantages of Different
Item Formats
64
Multiple-Choice Advantages
  • Possible to cover broad range of knowledge in
    limited time
  • Skillfully written items can measure higher-order
    cognitive skills
  • A MC test can be scored quickly and objectively
  • Strengths and weaknesses can be detected easily

65
Multiple Choice Disadvantages
  • Individual differences in skill at such items
    equals bias
  • Measure recognition rather than recall
  • More likely to measure recognition of content
    than integration or application of information
  • It is difficult and time-consuming to write good
    items
  • It is possible to assess higher-order cognitive
    skills, but most assess only knowledge
  • Some correct answers can be guesses

66
True/False Advantages
  • Same as multiple-choice, i.e.
  • Possible to cover broad range of knowledge in
    limited time
  • Skillfully written items can measure higher-order
    cognitive skills
  • A T/F test can be scored quickly and objectively
  • Strengths and weaknesses can be detected easily

67
True/False Disadvantages
  • Same as multiple-choice
  • In addition, often too "black white" or clear
    cut for subject matter (psychology often is not
    so clear cut
  • random guessing equals 50 correct

68
Short Answer and Completion Items Advantages
  • Requires recall of ideas
  • Can measure integration and application more
    easily than multiple-choice
  • Takes less time per item than essays (advantage
    of broader sampling)

69
Short Answer and Completion Items Disadvantages
  • Moderately time consuming
  • Less breadth of sampling than multiple-choice
  • Consistent scoring is difficult and time consuming

70
Essay Questions Advantages
  • Best measure of organizing, integrating, and
    evaluating information
  • Can approximate application of knowledge, easy to
    write

71
Essay Questions Disadvantages
  • Time consuming (reduces breadth of sampling)
  • Scoring is very time consuming
  • Scoring consistently is very hard
  • Writing skill and speed are often major sources
    of bias

72
III. Writing Exam Questions
73
Writing Multiple-Choice Items
  • Stem should present the problem, including
    qualifying statements
  • There should be only one correct answer
  • Distractors should be plausible but clearly
    incorrect
  • Avoid negative wording (especially double
    negatives)
  • Use "all of the above," "none of the above" and
    "A and B" sparingly
  • When item is controversial, indicate whose
    opinion is sought

74
Writing Multiple-Choice Items
  • Avoid irrelevant cues to correct answer (length,
    grammar)
  • Items should test one central idea or concept
  • Watch out for patterns in alternatives (overuse
    of "C" as correct answer)
  • Choose appropriate level of difficulty--best to
    write item figuring 50-75 of students would
    choose the correct answer
  • Assure even coverage of material and types of
    knowledge

75
More on MC
  • Multiple-choice questions seem to be many
    teachers' preferred testing method in that they
    are relatively easy to correct and grade
  • The construction of good MC questions does
    require some thought and planning The following
    are simple guidelines

76
Writing True-False Items
  • Choose words with precise, definite meanings
  • Avoid tricks and trivia
  • Avoid "always" and "never" (easy clues)
  • Follow guidelines for multiple choice

77
Writing Matching Items
  • Keep each matching set short (no more than 5
    stimuli/responses)
  • Each item should have only one correct match
  • Use longer items as stimuli, shorter as responses
  • Arrange responses in alphabetical or logical order

78
Writing Short Answer and Completion Items
  • Phrase item to elicit one correct answer (makes
    scoring possible)
  • Clearly indicate type of answer you want
  • Do not use more than two blanks per completion
    item
  • Try to place blank near the end of a completion
    item
  • Make a key before scoring
  • Periodically re-score early tests to detect
    shifting criteria

79
Writing Essay Questions
  • Limit questions to vitally important material
  • Clearly define task, scope, and directions for a
    "good" answer
  • Allow time for thought
  • Use multiple medium-length essays rather than one
    long one

80
Writing Essay Questions
  • Use questions that have a limited number of good
    answers
  • Allow choice between alternatives (e.g., "answer
    3 of the 4 questions")
  • Make a key before scoring
  • Periodically re-score early tests to detect
    shifting criteria

81
Presenting the question or problem
  • The 'stem' of a MC question is the statement for
    which a choice of response is to be made The stem
    has to be
  • Clearly stated
  • Composed of complete statements
  • Comprehensive enough so that the subsequent
    choices can be brief
  • Succinctly stated

82
Examples of well-stated stems
  • The term 'Native Americans' refers to which of
    the following groups?"
  • The US president's election is determined by"

83
Examples of poorly stated stems
  • Fluctuations in the stock market…
  • Unsupervised school children…

84
Determining response options
  • Typically, MC questions will include 4-5 response
    options To be effective, response options should
  • Include only one completely correct and
    defensible option
  • Include only choices that are logically connected
    to the stem
  • Include choices that represent misconceptions
    previously expressed by students
  • Include choices that are mutually exclusive
  • Avoid 'all of the above' or 'none of the above'

85
Formatting of response options
  • Approximate uniform length of items
  • Use uniform grammatical structures (all
    sentences, all phrases, etc)
  • Use a consistent number of options for all
    questions
  • State options positively, whenever possible
  • Distribute the correct option randomly among the
    choices (not all correct answers should be 'B',
    etc)
  • Eliminate verbal cues flowing directly from the
    stem

86
Formatting the Test
  • Are your directions CLEAR and CONCISE?
  • Are your objectives clear?
  • Are you being specific in the area you are
    covering?
  • Are your answers simple and factual?
  • Are your questions on the left and numbered?

87
Formatting the Test
  • Are your answers on the right and lettered?
  • Are your answers in some specific order? Ex.
    Alphabetical or Chronological
  • Is there a logical connection between questions
    and answers?
  • Are there more answers than questions?
  • Are your answers and questions concise?

88
Top Ten Rules for Writing Good Questions
  • Remember your tests purpose
  • If in doubt, throw it out
  • Keep your questions simple
  • Stay focused - avoid vague issues
  • If a question can be misinterpreted, it will be

89
Top Ten Rules for Writing Good Questions
  • Include only one topic per question (avoid
    "double-barreled" questions)
  • Avoid leading questions
  • Consider alternate ways to ask sensitive
    questions
  • Make sure the respondent has enough information

90
Things to remember when writing an essay test
91
Do your essay tests
  • Have Clear and Specific Directions
  • Contain Any Ambiguous Words
  • Have a Specific Task for Each Question
  • Avoid Factual Knowledge Questions
  • Avoid Giving Students Options
  • Account for Class Size and Time

92
  • MC questions that ask for the best answer out of
    choices that incorporate other answers can be
    confusing and should be limited

93
Punctuation
  • The MC question should be constructed so that the
    stem, when complemented by the option, makes up a
    complete, correct sentence
  • If the stem of a MC question is a question, then
    each option should begin with a capital letter 
  • Each option should not have a period unless it is
    a complete sentence in and of itself

94
Punctuation
  • If the stem is a question or statement with a
    blank in it that one option is intended to fill,
    all options should be lowercase with no period
    after them
  • If the stem is an incomplete question, and one
    option is meant to be added to the end in order
    to complete the sentence, the options should be
    lowercase and have a period after each
About PowerShow.com