Validity and Reliability - PowerPoint PPT Presentation

Loading...

PPT – Validity and Reliability PowerPoint presentation | free to view - id: 6d7aa1-YTkxY



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Validity and Reliability

Description:

Validity and Reliability Generate Hypotheses Theory: Alcohol impairs driving ability Step one generate testable hypotheses: Hypothesis 1: Heavier drinkers take ... – PowerPoint PPT presentation

Number of Views:147
Avg rating:3.0/5.0
Slides: 42
Provided by: elizab201
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Validity and Reliability


1
Validity and Reliability
2
Generate Hypotheses
  • Theory Alcohol impairs driving ability
  • Step one generate testable hypotheses
  • Hypothesis 1 Heavier drinkers take more risks
    while driving than lighter drinkers
  • Hypothesis 2 Alcohol slows down reaction time
    which causes more accidents
  • Hypothesis 3 Alcohol increases risk-taking while
    driving which causes more accidents

3
Testing Hypotheses
  • Step 2 We conduct a controlled test
  • We select a research design
  • We identify the population and select a sample
  • We define the variables of interest

4
Research Design
  • Non-experimental vs Experimental designs
  • Non-experimental observe a single group of
    subjects at one point in time
  • Observational studies
  • Survey Studies
  • Experimental involves multiple groups or
    multiple observations across time
  • Control standard against which the effects of
    the experimental condition is compared

5
Validity
  • Scientific Validity refers to the concept of how
    correct is my statement or do I trust my
    results.

6
Types of Validity
  • External Validity
  • Can we generalize our findings from the
    experimental context to other people, in other
    places, at other times, in other situations?

7
External Validity
  • Random Sampling
  • Identify Population
  • All people
  • All adults
  • All adults who drink alcohol
  • All adults who drive after drinking alcohol
  • Select Sample
  • Select sample from population of interest
  • Random
  • Stratified representative of the population on
    key characteristics

8
External Validity
9
Types of Validity
  • Internal Validity
  • Can we be confident that the observed outcomes
    are due to (caused by) our experimental treatment
    and NOT to some other cause?
  • How sure are we that manipulation of the
    independent variable caused the differences
    observed in the dependent variable

10
Internal Validity
  • If some other factor could have caused the
    results (BAC just accidentally covaries with risk
    taking) this is known as a confound
  • Confound uncontrolled and/or unmeasured
    characteristic(s) that accounts for the observed
    findings

11
Research Design
  • Random Assignment
  • Place subjects at random into different control
    and experimental conditions
  • Goal is to ensure that potential confounds are
    equally represented in both groups
  • Not always possible, so how do we ensure internal
    validity?

12
Internal Validity
  • In our study, we put all of the heavy drinkers in
    the group that received alcohol and all of our
    light or non-drinkers in the group that received
    water
  • To help subjects get in the mood, the bartender
    played loud and upbeat music during the drink
    alcohol condition. No music was played during
    the drink water condition.

13
Testing Hypotheses
  • Observation a man leaves a bar and gets into a
    car accident
  • Theory Alcohol impairs driving ability
  • Hypothesis 1 Heavier drinkers take more risks
    while driving than lighter drinkers
  • Hypothesis 2 Individuals who are intoxicated
    take more risks while driving than individuals
    who are sober

14
Testing Hypotheses
  • Independent variable
  • Predictor Variable
  • Variable that is manipulated
  • Dependent Variable
  • Predicted variable
  • Variable that depends on or is affected by the
    independent variable
  • Operational definition
  • Concrete description of how your variables will
    be measured

15
Testing Hypotheses
  • Hypothesis 1
  • IV level of drinking is defined using the
    quantity frequency index
  • DV Risk-taking is defined as
  • frequency of speeding (i.e., number of days per
    week that the individual drives 10 miles over
    the speed limit)
  • Frequency of tailgating

16
We analyze the data and reject or accept our
hypothesis
17
Statistical Analysis
18
Statistical Analysis
19
Statistical Analysis
20
Statistical Analysis
21
Statistical Analysis
Variance Explained
22
Pirates and Global Warming
23
Testing Hypotheses
  • Hypothesis 2
  • IV Intoxication is defined as a BAC of .08 mg
    or greater
  • DV Risk-taking is defined through use of a
    simulated driving task as
  • amount of time spent speeding and
  • number of times passing cars on a double yellow
    line

24
Statistical Analysis
25
Statistical Analysis
26
Construct Validity
  • Related to external validity
  • How well do my variables reflect what I want to
    measure
  • How good are your operational definitions

27
Reliability
  • Good science means hypotheses are not only
    testable but REPEATABLE

28
How to ensure reliability
  • Test retest reliability
  • One person can make the same observation several
    times
  • Do they exhibit temporal consistency?
  • Inter-rater reliability (inter-observer
    agreement)
  • Do several judges or raters come to similar
    conclusions

29
Internal Consistency
  • How well do items that should be related score
    similarly
  • Items that are similar on a scale should covary.

30
Reliability
  • More is better
  • 10 ratings is more reliable than 5
  • 20 ratings is more reliable than 10
  • 5 Raters (judges) are better than 1
  • 10 Raters (judges) are better than 5

31
Measurement
  • In general the more reliable the measure the more
    valid the results
  • Type of measurement scale can influence the
    reliability of your results

32
Measurement Scales
  • Nominal Scale or Categorical Scale
  • You score one or the other
  • Examples
  • Urine test positive or negative
  • Sex male or female
  • Pregnant yes or no
  • Shoe type loafer, tennis shoe, hiking boot

33
Measurement Scales
  • Ordinal scales
  • Ordinal scales rank order things but tell you
    nothing about the distance between them
  • Examples
  • Foot race 1st, 2nd, 3rd, etc
  • Olympic Medals Gold, Silver, Bronze

34
Measurement Scales
  • Interval Scales
  • Interval scales have numerical ordering
  • Many Psychological tests use interval scales
  • Examples
  • IQ
  • SAT
  • More than one person can have the same score
  • Distance between intervals not necessarily the
    same
  • SAT is the difference between 560 and 600 the
    same as the distance between 760 and 800

35
Measurement Scales
  • Ratio Scales
  • All intervals have the same amount between them
  • Examples
  • Difference between 100 kg and 110 kg is the same
    as the difference between 40 and 50 kg
  • Can express as ratios
  • Example
  • 40kg is half as heavy as 80 kg
  • 20 kg is half as much as 40 kg

36
Measurement Scales
  • General rules of scales
  • The better the scale the easier it is to get
    reliability
  • You can convert ration scales to interval to
    ordinal to categorical but you cannot go in the
    reverse
  • Make your scales as reliable as possible

37
Measurement Scales
  • Example of general rules
  • You versus Tiger Woods in golf
  • Measurement Scale categorical
  • number of holes in one (yes hole in one, no,
    missed the cup)
  • May take thousands of shots to determine who is
    the better golfer (who gets more holes in one on
    the long run)

38
Measurement Scales
  • Example of general rules
  • You versus Tiger Woods in golf
  • Measurement Scale Ordinal
  • Rank order shots closest to the hole
  • You Tiger
  • 25ft 27ft
  • 100 ft 23ft
  • 30 ft 32 ft
  • 125 ft 27 ft
  • 65 ft 29 ft
  • Better but still harder to represent truth

39
Measurement Scales
  • Example of general rules
  • You versus Tiger Woods in golf
  • Measurement Scale Interval or Ratio
  • Rank measure the distance of the shots and get
    the total feet from the cup
  • You Tiger
  • 25ft 27ft
  • 100 ft 23ft
  • 30 ft 32 ft
  • 125 ft 27 ft
  • 65 ft 29 ft

40
Measurement Scales
  • Test Measurement scales can affect reliability
  • True false less reliable than ratings on a 5
    point likert scales
  • 9 point likert scale more reliable than a 5 point
    scale
  • Up to a point - adding more doesnt always add
    more, but it doesnt hurt (13 not necessarily
    better than 9, but never worse)

41
Measurement Scales
  • Test Measurements can affect reliability
  • Example 6 point scale of TV watching
  • 1 6
  • up to 0.5 hrs more than 2.5 hrs
  • 1 6
  • Up to 2.5 hrsmore than 4.5 hrs
About PowerShow.com