From Abstract to Concrete: Operationalization and Measurement - PowerPoint PPT Presentation


PPT – From Abstract to Concrete: Operationalization and Measurement PowerPoint presentation | free to view - id: 86dfb-ZDc1Z


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

From Abstract to Concrete: Operationalization and Measurement


Our comparisons can be accurate only to the extent that the ... Comparing scores on measure being validated with scores on measures from other variables ... – PowerPoint PPT presentation

Number of Views:110
Avg rating:3.0/5.0
Slides: 31
Provided by: www43


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: From Abstract to Concrete: Operationalization and Measurement

From Abstract to Concrete Operationalization
and Measurement
  • Sharon Paynter
  • Spring 2007
  • Manheim Ch 5

Formulating Theory
  • Empirical Research is a means of obtaining
    answers to questions about our observations
  • Theory often stated in abstract terms
  • Answers we want are concrete and specific
  • How can we quantify our concepts in order to make
    precise statements about whether or not our
    theory is supported by our observations?

Processes and definitions
  • Operationalization
  • Process of selecting observable properties to
    represent abstract concepts
  • Instrumentation
  • Specific steps to take in making observation
  • How something will be measured precise,
    standardized indications of extent to which
    characteristic is present
  • Measurement
  • Result of applying an instrument to assign
    numerical values to cases
  • Evidence used in making decisions and answering
  • Observation
  • Applying a measuring instrument in order to
    assign values for some characteristic of the
    phenomenon to the cases being studies

  • We can never actually compare concepts, what we
    compare are indicators of concepts
  • Our comparisons can be accurate only to the
    extent that the indicators selected mirror the
    concept we intend them to measure (validity)
  • Improper operationalization poor reflection of
  • ? faulty conclusion

Example Terms
  • Want to test the impact of poverty on educational
  • Concept Educational achievement
  • Variable Class rank
  • Indicator Test scores
  • Values Numerical values from 0 to 100
  • Used to compare groups who are impoverished to
    those who are not

Multiple Indicators
  • Most social science concepts are multidimensional
    (more than one aspect or component)
  • Our measures should reflect the diversity of the
    concept if they are to be useful indicators.

Multiple Indicators Examples
  • Corn
  • Height
  • May be no difference in height, may also examine
  • Stalk width
  • Leaf size
  • Corn Yield
  • Weight of corn ears
  • Democracy
  • Hold Regular Elections
  • Include Iraq under Saddam Hussein
  • What else should we examine to get at what we

Operational Definitions
  • Specifying a set of procedures to obtain an
    empirical indicator of a concept in any given
  • Must tell us precisely and explicitly what to do
    in order to determine what quantitative value
    should be associated with a variable in any given

Operational Definition Importance of Precision
  • Tell others exactly what we have done
  • Evaluate or replicate study
  • Want all measurements collected in exactly the
    same way
  • Research assistants, data collectors
  • If differences in data collection, results will
    not be comparable invalid conclusions
  • Precise and detailed statements of how to
    operationalize a variable will help us in
    evaluating the results we obtain and in
    eliminating rival explanations

Operationalization Example
  • Party unity
  • Voting together on roll call votes
  • Which votes to use
  • Procedure for determining how majority of party
    voted on each issue
  • How do we treat abstentions?
  • Procedure for computing and then averaging
    percentages of agreeing votes for each legislator

Developing an Instrument
  • Operational definition results in an instrument
    for taking measurements
  • Examples include
  • Series of questions on survey
  • Instructions for observing a certain event (e.g.
    UN debate)
  • Sets of numbers to be taken from sourcebook and
    rules for combining them into a measure

  • Result of applying an instrument to assign
    numerical values to cases
  • Evidence used in making decisions and answering

Level of Measurement
  • Classification according to how much information
    it gives us about the phenomena being measured
    and their relationship to one another
  • Nominal
  • Ordinal
  • Interval/Ratio

Levels of Measurement
  • Nominal
  • Set of discrete categories to distinguish between
  • Naming or classifying cases into groups
  • Only allows sorting cases into groups
  • Mutually Exclusive each case can only be
    assigned to a single category
  • Collectively exhaustive all cases can be
    assigned to some category
  • Party Affiliation Democrat, Republican,

Level of Measurement
  • Ordinal
  • Allow us to associate number with each case
  • Categorize, order and rank cases
  • Relative differences
  • Social Class Lower, Middle, Upper
  • Attitude Strongly Disagree, Agree, Neutral,
    Disagree, Strongly Agree

Level of Measurement
  • Interval/Ratio
  • Exact differences between cases
  • Associate a number with each case
  • Standard unit of property being measured
  • Zero Point
  • Interval 0
  • Ratio Absolute 0 (nothing below)
  • Age 0 - ???

Ordinal vs Interval/Ratio
  • Under 10,000
  • 10,00019,999
  • 20,000-29,999
  • 30,000-39,999
  • 40,000-49,999
  • Difference in categories 1-10,000
  • Exact salary
  • Open ended question
  • Provide exact comparison between salaries
  • 10,000 ½ of 20,000

Stronger Measurement Better
  • You want operationalizations that allow interval
    level measurement whenever possible and
  • Interval provides most information and allows
    mathematical calculations
  • If you use lower level of measurement you may
    be wasting potentially valuable information (you
    can always combine data to make a stronger
  • Republican, Democrat, Independent
  • Strong to weak affiliation
  • Level of measurement should be
  • Theoretically defensible
  • Technologically possible (measurement technology)

Caveat Measurement Precision
  • Cases where too much precision in measurement is
  • Age and participation in 2002 election
  • Giving up some precision may provide clearer
    results (e.g. 5 yr groupings)
  • Operationalize concepts as precisely as possible
  • Can collapse categories if necessary

Outcome of Measurement
  • 2 sources of differences in scores from a
  • Real Differences Actual variation in property
    we are examining
  • Measurement Error Differences in values
    assigned to cases that can be attributed to
    anything other than real differences
  • Measure or setting which causes differences
  • Do NOT reflect authentic differences in
    properties we are examining

Systematic Measurement Error
  • Constant among cases and studies in which same
    measure is used
  • Confusion in variables
  • Nature of instrument
  • Results invalid
  • Differences (or similarities) our measure reveal
    are not accurate reflections of differences we
    think we can measure

Random Measurement Error
  • Affects each application of instrument
  • Matter of chance due to
  • Transient characteristics in our cases
  • Situational variations in application of
  • Mistakes in administration and processing
  • Other factors that vary from one use of
    instrument to the next
  • Makes measures unreliable
  • Cannot consistently get same results

Measurement Error Distorting Influences
  • Differences in distribution of other, relatively
    stable characteristics among the cases that are
    unintentionally revealed by our measures
  • Political Ideology Intelligence/Region/Culture
  • Differences in the distribution of temporary
    characteristics among the cases that are
    reflected in our measures
  • Mood, health, events (corruption, disaster)
  • Differences in subjects interpretation of the
    measuring instrument
  • Ambiguously worded (vote in last electionwhich
  • Differences in the setting in which the measure
    is applied
  • Race, sex, age of interviewer

Measurement Error Distorting Influences
  • Differences n the administration of the measuring
  • Differences in scores as a result of errors that
    occur during data collection/recording
  • Interviewer misinterprets instructions, poor
    lighting, broken pencils etc.
  • Differences in the processing analysis of data
  • Differences in the way individuals respond to the
    form of the measuring instrument

Validity Are we measuring what we think we are
  • Extent to which our measures correspond to
    concepts they are intended to reflect
  • Properties of a valid measurement
  • Appropriate Describes a concept suitably
  • Complete Includes correct properties
  • Extent to which differences in scores on a
    measure reflect only differences in the
    distribution of values on the variable we intend
    to measure
  • Main concern is systematic error
  • Depends on knowledge of subject and careful
    analysis of alternative operationalizations
  • Can only be tested after we have collected data

Testing Validity
  • Pragmatic or Predictive Validity
  • Assessing validity of a measure from evidence of
    how well it works in allowing us to predict
  • Requires alternative indicator of variables to
    check measures
  • Face Validity
  • On the face of it are there good reasons to
    think that this measure is an accurate gauge of
    the intended characteristic?

Testing Validity
  • Construct Validity
  • Extent to which actual relationships between
    scores of various measures are consistent with
    what we expect from our theory
  • External Validity
  • Comparing scores on measure being validated with
    scores on measures from other variables
  • Strength of Alliance Voting together in UN
    Number of trade barriers

Testing Validity
  • Internal/Convergent Validity
  • Comparing scores on various measures of SAME
  • Quality of Street Lighting survey of residents,
    light meter, independent rating, rating pictures
    of other streets
  • Multiple Indicators allows us to test validity of
    measures and increases of obtaining a valid
    measure in first place
  • Discriminant Validity
  • Comparing scores on measures that represent
    different/ opposite concepts

  • How stable are the values from a measure
  • If not reliable not valid
  • If reliable can still be invalid
  • Measure may be reliable without being valid, but
    cannot be valid without being reliable
  • Only subject to random error

Testing Reliability
  • Test-Retest Method
  • Same measure is applied to same set of cases
    multiple times over time
  • Alternative Form Method
  • Different forms of measurement applied to same
    group at the same time
  • Sub sample Method (split-half)
  • Draw one sample divide it into several sub
  • Give same measure to all sub samples
  • Compare answers from different sub samples

  • Used to test data collection procedures
  • Reliability and validity must be established
    before beginning a study