Methods of Measuring Behavior

1
Chapter 6

• Methods of Measuring Behavior

2006 Prentice Hall, Salkind.
2
CHAPTER OVERVIEW

• Tests and Their Development
• Types of Tests
• Observational Techniques
• Questionnaires

3
SOME IMPORTANT FIRST CONSIDERATIONS

• Is the tool you propose to use reliable and
  valid?
• Base your choice of research tools on how you
  have asked the research question

4
TESTS AND THEIR DEVELOPMENT

• A test should measure the nature and extent of
  individual differences
• A good test differentiates people based on their
  true scores

5
WHY USE TESTS?

• To help determine outcomes of experiments
• To diagnose strengths and weaknesses
• To assist in placing individuals in appropriate
  programs
• To assist in selecting applicants
• To evaluate a programs effectiveness

6
TYPES OF TESTS Overview
7
ACHIEVEMENT TEST SOURCES

• Standardized
• Commercially prepared for wide use
• Scoring instructions included
• Researcher-made
• Designed by user for specific purpose
• Scoring instructions specific to test

8
ACHIEVEMENT TESTS REFERENCING What Comparison Do
Tests Make?

• Norm-referencedIndividuals scores are compared
  to results from a larger group
• Criterion-referencedIndividuals scores are
  compared to defined performance standards

9
MULTIPLE-CHOICE ACHIEVEMENT ITEMS Anatomy of a
Multiple-Choice Item
10
TO USE OR NOT TO USE?

• Advantages
• Versatile
• Easy to score
• Simple to take
• Poor writers not penalized
• Good items used again
• Good distracters are diagnostic
• Hard to fake correct answer

• Disadvantages
• Limit students options
• No opportunity to practice writing
• Some people dont do well on them
• Limits content to be assessed
• Items must be well written

11
ITEM ANALYSIS HOW TO TELL IF YOUR ITEMS WORK

• Questions should discriminate those who know the
  material from those who dont
• Item analysis provides two measures of a
  questions ability to discriminate
• Difficulty index
• Discrimination index

12
COMPUTING INDICES First Steps
     

• Rank scores from highest to lowest
• Choose top 27 of scores for high group
• Choose bottom 27 of scores for low group

13
COMPUTING INDICES
14
COMPUTING INDICES

• Difficulty index
• Proportion who answered item correctly
• D NCh NCl
• T

• Discrimination index
• Proportion in high group who answered correctly
  minus proportion in low group who answered
  correctly
• D NCh - NCl
• (.5)T

15
RELATIONSHIP BETWEEN ITEM DISCRIMINATION AND
DIFFICULTY
Perfect Discrimination When ½ gets item
right, ½ gets item wrong
½ right is in upper half,
½ wrong is in lower half
16
ATTITUDE TESTS Assess an Individuals Feelings
About a Topic
17
THURSTONE SCALES Method of Equal Appearing
Intervals

• Statements indicating an attitude toward a topic
  are written
• Judges rank the statements from least favorable
  to most favorable
• Statements receiving consistent ratings are given
  the average score
• A set of statements is selected that covers the
  entire range of attitudes

18
THURSTONE SCALES Administration

• Respondents check items with which they agree
• Well-formed attitudes are indicated by
  consistently checking either high or low items
• Poorly-formed or inconsistent attitudes are
  indicated by inconsistent pattern or by checking
  off many neutral items

19
LIKERT SCALES

• Statements indicating an attitude toward a topic
  are written
• Items with clearly positive or negative attitudes
  are selected
• Statements are listed with a space for respondent
  to indicate degree of agreement

20
A LIKERT SCALE
21
SCORING LIKERT RESPONSES Method of Summated
Ratings

• Items are weighted
• Weights of unfavorable items are reversed
• Average score is computed

       
       
       
22
PERSONALITY TESTS

• Projective tests
• Present respondent with ambiguous stimulus
• Structured tests
• Questions are objective

23
OBSERVATIONAL TECHNIQUES

• Researcher observes and records
• Does not interfere with behavior

24
TECHNIQUES FOR RECORDING BEHAVIOR
25
OBSERVATIONAL TECHNIQUES? BE CAREFUL! Pitfalls
to Avoid

• Observer effects
• Observer bias
• Fatigue
• Changing definitions

26
QUESTIONNAIRES

• What they are
• Paper and pencil tests with structured questions
• Self-administered

27
QUESTIONNAIRES

• Advantages
• Can be mailed out
• Survey broad geographic area
• Cheaper than one-on-one interview
• Respondents may be more honest
• Data easy to share with other researchers
• Disadvantage
• Low return rate

28
BASIC ASSUMPTIONS OF A QUESTIONNAIRE

• Does not make unreasonable demands upon the
  respondent
• Does not have a hidden purpose
• Requests information that respondents presumably
  have
• Contains interesting questions
• Does not request information that could be
  obtained by other means

29
THE QUESTIONS

• The questionnaire contains questions that can be
  answered
• The questionnaire contains questions that are
  straightforward

30
THE FORMAT

• The presentation is attractive, professional, and
  easy-to-understand
• Questions and pages are clearly numbered
• Directions are clear and explicit
• Questions are objective
• Questions are ordered sensibly
• Transitions are used from one topic to the next

31
THE IMPORTANCE OF USING A COVER LETTER

• Informs the recipient about the research
• Establishes the importance of the research
• Makes the recipient a part of the research