Reliability

1
Reliability Validity

• Reliability
• dependability
• is the indicator consistent?
• same result every time?
• Does not necessary measure what you think it
  measures (ex. May consistently measure something
  other than the concept)
• Validity
• measurement validity - how well the conceptual
  and operational definitions mesh with each other
• does measurement tool measure what we think ?

2
Relationship between Measurement Reliability and
Validity

• reliability necessary for validity but does not
  guarantee it (necessary but not sufficient)
• measure can be reliable but invalid

3
Types of Reliability

• stability
• over time
• representative
• across different subgroups of a population
• (ex. of problems young people may exaggerate
  their ages, older people may reduce theirs)
• equivalence
• Comparable results from multiple indicators
• intercoder reliability

4
Improving Reliability

• clearly conceptualize constructs
• increase level of measurement
• use pretests, pilot studies
• e.g. use multiple indicators

Measurement Using Multiple Indicators Neuman
(2000 167)
5
Selected Types of Validity Face Validity

• judgement by the scientific community that
  indicator measures the concept

6
Content Validity

• measure represents all the aspects of conceptual
  definition of construct.
• judgement concerning how adequately a test
  samples behavior representative of the universe
  of behavior the test was designed to sample.

Love
7
Criterion Validity

• The validity of an indicator is verified by
  comparing it with another measure of the same
  construct in which a researcher has confidence.
• Predictive and concurrent validity

8
Construct Validity

• A type of measurement validity that uses multiple
  indicators
• convergent and discriminate

9
Other Dimensions of Validity

• Internal Validity
• no error of logic internal to research design
• External Validity
• results can be generalized
• Statistical validity
• correct statistical methodology chosen ?
• assumptions fully met

10
More terms concepts for assessing validity
(usually in experimental research)

• Threats to Internal Validity
• Selection bias
• History effects (something in context changes)
• Maturation
• Testing effect
• Instrumentation
• Mortality
• Statistical regression
• Contamination
• Compensatory behaviour
• Experimenter expectanc
• Threats to External Validity
• Realism
• Reactivity

11
Another Aspect of Measurement Design Levels of
Measurement

• 1-Nominal (ex. Mother tongue)
• different categories (names, labels, images)
• not ranked
• 2-Ordinal (county fair prizewinners ranked by
  first, second third prize)
• different categories
• rank-ordered
• attributes indicate relatively more or less of
  that variable
• distance between the attributes of a variable is
  imprecise

12
Levels of Measurement (contd)

• 3- Interval Measures (age measured by 5 year age
  groups)
• different categories
• ranked in order
• Can tell amount of difference between categories
• Usually no true zero
• 4- Ratio Measures (age measured by date of birth)
• different categories
• ranked in order
• amount of difference between categories
• also possible to state proportion (have a true
  zero)
• Relations between levels --can collapse from
  higher into lower, not vice versa

13
The Research Process
Babbie (1995 101)
14
(No Transcript)