Quantitative Methods in Social Research 2010/11 Week 3 (morning) session 28th January 2011 Data Sources: Secondary Analysis, Official Statistics - PowerPoint PPT Presentation


PPT – Quantitative Methods in Social Research 2010/11 Week 3 (morning) session 28th January 2011 Data Sources: Secondary Analysis, Official Statistics PowerPoint presentation | free to download - id: 505918-NTk3N


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

Quantitative Methods in Social Research 2010/11 Week 3 (morning) session 28th January 2011 Data Sources: Secondary Analysis, Official Statistics


Quantitative Methods in Social Research 2010/11 Week 3 (morning) session 28th January 2011 Data Sources: Secondary Analysis, Official Statistics & – PowerPoint PPT presentation

Number of Views:310
Avg rating:3.0/5.0
Slides: 42
Provided by: rachella3
Learn more at: http://www2.warwick.ac.uk


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Quantitative Methods in Social Research 2010/11 Week 3 (morning) session 28th January 2011 Data Sources: Secondary Analysis, Official Statistics

Quantitative Methods in Social Research
2010/11 Week 3 (morning) session 28th January
2011 Data Sources Secondary Analysis,
Official Statistics Statistics without
What is secondary analysis?
  • Hakim any further analysis of an existing
    dataset which presents interpretations,
    conclusions or knowledge additional to, or
    different from, those presented in the first
    report on the inquiry as a whole and its main
  • Dale et al. secondary analysis implies a
    re-working of data already analysed.
  • Hyman the extraction of knowledge on topics
    other than those which were the focus of the
    original surveys.
  • Online course extracts Dale et al. 1988 Dale et
    al. 2008

Sources for secondary analyses
  • Surveys
  • The Census
  • Administrative and/or public records
  • Longitudinal studies
  • Qualitative studies
  • The UK Data Archive (http//www.data-archive.ac.uk
    ) now catalogues data from surveys and
    qualitative studies, as well as the Census,
    historical data, international country-level
    databases, etc.

Some sources specifically geared towards
secondary analysis
  • The British Social Attitudes Survey
  • Understanding Society http//www.understandingsoci
  • The Timescapes qualitative longitudinal study

Benefits of secondary analysis
  • It avoids costs in money and time that would make
    primary research impractical, especially for a
    lone researcher.
  • It allows one to benefit from the fieldwork
    expertise of professional organizations.
  • Cross-national and historical research become
    more of a practical possibility.
  • Secondary analyses of longitudinal data
    facilitate studies of change over time.
  • Large, nationally representative samples
    facilitate sophisticated, generalisable analyses,
    and (sometimes) analyses relating to
    small/relatively inaccessible minorities.

Bridging the quantitative/qualitative divide
  • Dale et al. comment that qualitative research
    can greatly enhance the value of secondary
    analysis by providing greater depth of
    information, particularly by suggesting the
    underlying processes that are responsible for the
    observed relationships.

Why doesnt sociology in the UK generate more
secondary analyses?
  • A lack of quantitatively-orientated researchers.
  • The legacy of critiques of quantitative research
  • More specifically, the legacy of critiques of
    official statistics.
  • Although its now more of a question of inertia
    than of ongoing scepticism?

Themes within critiques of official statistics
  • Concerns about coverage
  • Concerns about measurement
  • Epistemological concerns
  • Political concerns

A damning quote?
  • Its i.e. the states economic and political
    functions are embedded in the production of
    official statistics, structuring both what data
    are produced and how this is done... only by
    understanding that statistics are produced as
    part of the administration and control of a
    society organised around exploitative class
    relations can we grasp their full meaning
  • (Miles and Irvine, 1979).

  • Analyses of official data can produce
    substantively interesting results
  • The producers and users of official statistics
    are normally very concerned about the errors in
    data and the datas limitations,
  • The conceptual issues arising from the use of
    official statistics are not dissimilar to those
    arising in other forms of sociological research.
  • Analyses of official data have been used to
    critique governments with respect to issues such
    as unemployment, health inequalities, etc.
  • (The first three of the above bullet points are
    suggestions by Bulmer)

... nevertheless
  • As Hindess commented
  • Official statistics are never mere givens to be
    taken as they are or else dismissed as
    inadequate. Like other productions they must be
    explained in terms of the conditions and
    instruments of their production.
  • As structured social products they i.e.
    official statistics can and should be! be
    critically assessed.

Official statistics or official data?
  • Published official statistics have justifiably
    been viewed with some scepticism.
  • However, the analysis of official data by a
    secondary analyst can avoid some of the problems.
  • Given access to the raw official data, she or
    he can manipulate them in ways different to how
    they were processed to produce published official

Are UK official statistics getting more
  • A Statistics Board resulting from the Statistics
    Bill of July 2007, renamed the UK Statistics
    Authority in February 2008 (see
    http//www.statisticsauthority.gov.uk/) is
  • ... an independent body operating at arm's
    length from government as a non-ministerial depart
    ment, directly accountable to Parliament. 
    its overall objective is to promote and
    safeguard the quality of official statistics that
    serve the public good. It is also required
    to safeguard the comprehensiveness of official

Some more specific developments I
  • OPCS now ONS Disability Surveys were criticised
    for not adequately reflecting disabled peoples
    perspectives on their disabilities.
  • (see Abberleys chapter in Levitas and Guy,
  • However, they were nevertheless used for some
    interesting and useful secondary analyes (see
    Pole and Lampard, 2002, Ch. 7).
  • More recently, the Office for Disability Issues
    (ODI) brought together a group of disabled people
    as a reference network, in part to facilitate the
    effective design of a new longitudinal disability
    survey, the Life Opportunities Survey (LOS)
  • (see http//www.ons.gov.uk/about/surveys/a-z-of-s

Some more specific developments II
  • A number of UK government surveys now (since
    2009) ask a question on sexual orientation,
    following a question being asked in the 2007
    Citizenship Survey (and resulting from ONS's
    Sexual Identity Project, established in 2006).
  • This development reflects more general
    governmental concerns about the availability of
    equality data.
  • However, it does not seem that a question will be
    asked on this topic in the 2001 Census!
  • The consensus also seems to be that the results
    generated by the question will under-estimate
    non-heterosexual orientations.

What is the moral? Must have a moral
  • Whether the source of their data is official or
    non-official, secondary analysts should gain an
    extensive knowledge of the research design and
    data collection process.
  • This allows the secondary analyst to adopt an
    informed and suitably critical approach to their
    assessment of the validity and value of their
    data source(s).
  • The data source for the slide title (I think) is
    A Funny Thing Happened on the Way to the Forum

Key issues in secondary analysis (according to
Dale et al.)
  • What was the original purpose of the study and
    what conceptual framework was used? Who was
    responsible for collecting the data?
  • What data did the study collect and how were
    variables such as occupational class
  • What was the sample design that was used and what
    was the level and pattern of non-response?

Is there documentation available in relation to?
  • Sample selection
  • Patterns of (non-)response
  • Interview schedules Questionnaires
  • Instructions to interviewers
  • The coding of answers
  • The construction of derived variables

Some other relevant questions...
  • Is secondary analysis an appropriate approach
    given the researchers objectives?
  • Does the secondary analyst know the topic area
    well enough to be able to interpret and evaluate
    the information available?
  • What similarities and differences are there
    between the conceptual frameworks of the original
    researchers and of the secondary analyst?
  • Are the data recent and extensive enough for the
    secondary analysts purposes?
  • How consistent is the information with
    information from other sources?
  • Is the information representativeness enough to
    support generalisations? Is weighting needed to
    correct for a lack of representativeness?

An example the General Household Survey
  • Advantages of the GHS include
  • a large sample size
  • the fact that it has been repeated more or less
    annually since the early 1970s, which allows
    trends to be examined
  • a broad agenda which means that relationships
    between concepts belonging to different policy
    areas can be examined
  • a hierarchical structure, which allows linkages
    between different members of the same household
    to be examined (Dale et al.)

Some examples of sources and issues from
Richards research
  • Social Change and Economic Life Initiative
    (SCELI) main survey (1986)
  • National Survey of Sexual Attitudes and
    Lifestyles II (2000) and various other
    couple-related surveys
  • General Household Survey (1991 2005)
  • British Election Study (1987)
  • See also Pole and Lampard, 2002, Ch. 7.

Reasons for the end of a cohabiting or marital
relationship (as shown on a NATSAL II showcard)
  • Unfaithfulness or adultery
  • Money problems
  • Difficulties with our sex life
  • Different interests, nothing in common
  • Grew apart
  • Not having children
  • Lack of respect or appreciation
  • Domestic violence
  • Arguments
  • Not sharing household chores enough
  • One of us moved because of a change in
  • (for example, changed jobs)
  • Death of partner
  • Another reason (please say what)

...and the categories that had to be added
  • Drink, drugs or gambling problem
  • Mental health or related problem
  • Problem with children/step-children
  • Never at home (e.g. always out with friends)
  • Problems with parents/in-laws/family
  • Age-related problems (e.g. big age difference)
  • Another relationship involved
  • Lived in/moved to a different country/area
  • Still in relationship, but stopped living
  • Change of mind/feelings/personality
  • Partner just left without any explanation

Statistics without surveys
  • There are methods of quantitative analysis that
    do not rely on surveys. Three that we will
    discuss are
  • Content analysis (to which Eric will return in
    Week 4)
  • Analysis of comparative-historical materials
  • Observation(al) studies
  • All involve the operationalisation of concepts
    and coding of data, as well as decisions about
    sampling and so none are immune from criticisms
    aimed at these processes, and the subjectivity
    involved therein.
  • But since all three largely involve unobtrusive
    methods, they tend not to involve the
    (artificial, potentially power-laden, and much
    criticised) interactions found in survey
  • We will conclude by looking briefly at network
    analysis. This is actually a particular method of
    statistical analysis, but one that has been
    developed in relative isolation to mainstream
    statistics, and one that has different starting
    assumptions and utilizes different sorts of data

Data Collection Qualitative vs. Quantitative
Qualitative Quantitative
Observing Participant observation Structured observation
Talking to people In-depth interviews Focus groups Surveys
Looking at texts (books, films, web pages, adverts) Discourse analysis Content analysis
Using existing information Comparative -historical research Analysis of existing statistics/data
Other Experiments / quasi-experiments (not common)
Note Network Analysis is largely quantitative,
but involves a whole set of different analytic
techniques data can be from surveys, structured
obs, or content analysis.
Content Analysis
  • Method of transforming symbolic content of a
    document (such as words or images) from a
    qualitative unsystematic form into a quantitative
    systematic form.

See Bryman, 2008, Ch. 12 (online course extract)
Possible Units of Analysis for Content Analysis
but a unit of analysis may also be a film, a
scene, a TV episode, a wall (containing
graffiti), a rubbish bin, a politicians speech,
a web-site, or a blog posting
Comparative-Historical Research
  • Much comparative-historical research does not use
  • However if you are looking at change over time or
    are comparing different countries or regions
    there are a large number of statistics that can
    be used
  • Macro-level secondary statistics e.g. World
    Bank development indicators i.e. mortalitity
    rates televisions per 1000 population Literacy
    rates. Or OECD Main Economic Indicators i.e.
    foreign direct investment GDP GNP etc.
  • See the Library Statistics Workbook that is
    linked to the module web page this is of value
    both in terms of accessing international data and
    statistical sources generally
  • Primary statistics these are datasets that you
    construct for yourself from historical and
    comparative research. They may document anything
    from the strength and political composition of
    particular trade unions in a particular time and
    place to land-holding patterns in different
    regions as described by local tax-records to
    speeches made by Vice-Chancellors of UK
    universities at public forums over the last
    century To conduct quantitative analysis of
    primary historical research it just needs to be
    systematically coded.

Sampling Comparative-Historical Events
  • If you are going to use comparative-historical
    data to create a dataset it is important to think
    about whether you have data from the entire
    population of events that you are interested in
    (i.e. every strike that occurred in the UK
    between 1990 and 2000), or whether you are
    focusing on a subset (thirty strikes that
    occurred in the UK between 1990 and 2000).
  • If you present statistical information for a
    subset of events you are sampling and the same
    issues of occur as any other time that you sample
    data your findings are only statistically
    generalisable if the sampling is random (or if
    each event has a known - typically equal -
    probability of selection into the subset).
  • On the other hand, there are often substantive
    reasons to choose specific important events to
    be part of your subset (i.e. large-scale strikes
    that involved media campaigns). This is
    legitimate and statistics gleaned from these may
    be interesting and informative. However they are
    not statistically generalisable to all events
    (i.e. strikes generally) and so inferential
    statistics are not appropriate.

Observation(al) studies
  • Observation is not just the preserve of
    qualitative methods. Quantitative methods can be
    applied where structured or systematic
    observation is carried out.
  • Like qualitative observation studies (and
    surveys), this involves cross-sectional data (we
    can only observe the present).
  • Unlike qualitative observation, structured or
    systematic observation is not inductive but
    requires the prior determination of what to
    observe (although this may be suggested by
    initial unstructured observations).
  • See Pole and Lampard, 2002, Ch. 4.

The observation schedule
  • To produce quantitative data an observation
    schedule or coding scheme is required.
  • This describes what is to be observed and how
    what is observed should be coded.
  • For example, if I were observing in the Library
    Café and was interested in interactions between
    students and the staff working at the
    cash-registers I could code each students
    behaviour in the following way
  1. No conversation, no eye contact, no smile
  2. Eye contact and/or smile, no conversation
  3. Conversation, only as required by the transaction
  4. Conversation as required by the transaction and
    polite thanks.
  5. Conversation that goes beyond transaction and
    polite thanks.

The observation schedule
  • The observations must be focused and relevant
    to the research question
  • The schedule (like closed questions in a
    questionnaire) should have categories that are
    mutually exclusive and exhaustive
  • Recording should involve as little observer
    interpretation as possible this is where
    reliability is diminished.

Sampling in Structured Observations
  • It is important to be clear about the unit of
    analysis are you sampling events/situations,
    interactions, or individuals?
  • Sampling must consider the dimension of time in
    determining who, where, and when to make
    observations. It may sometimes be appropriate to
    sample at multiple time periods and in multiple

Benefits and Drawbacks of Structured Observation I
  • Like other unobtrusive measures structured
    observation may avoid researcher contamination
    enabling the study of people in their natural
  • Unlike surveys it does not depend on the
    negotiation of meaning between interviewer and
    interviewee (or the interviewees accurate
    representation of her behaviour).
  • Unlike qualitative observation studies it can
    produce relatively reliable data and since
    observation (with a schedule) can be undertaken
    by more than one researcher, it enables
    large-scale data collection.

Benefits and Drawbacks of Structured Observation
  • However the researcher will only see the
    predetermined categories of action that the
    schedule specifies. These may not be the
    categories of action that are relevant to
  • Since structured observation precludes
    questioning participants about their motives or
    opinions, it is wholly dependent on observing
    behaviour and on the ability of the researcher to
    appropriately assess this.
  • It is ahistorical, in that it can only assess
    behaviour in the moment (unlike surveys which can
    ask, albeit imperfectly, about peoples pasts, or
    other methods such as content analysis,
    historical or secondary data analysis).

Network Analysis
  • Network Analysis is based on the assumption that
    peoples actions are interdependent and so it is
    critical to describe the networks of
    relationships that exist.
  • It is characterized by a distinctive methodology
    encompassing techniques for collecting data,
    statistical analysis, visual representation, etc
  • Critically, network analysis uses Matrices to
    analyse the relationships between people,
    organisations and institutions. It also uses
    graph theory.

Network analysis is concerned with attributes of
pairs of individuals, of which binary relations
are the main (but not only kind. Some examples
of dyadic attributes
  • Kinship brother of, father of
  • Social Roles boss of, teacher of, friend of
  • Affective likes, respects, hates
  • Cognitive knows, views as similar
  • Actions talks to, has lunch with, attacks
  • Flows number of cars moving between
  • Distance number of miles between
  • Co-occurrence is in the same club as, has the
    same colour hair as

The relationships neednt be between individuals
  • Ties could be
  • between corporations, or
  • between political organizations, or
  • between community groups, or
  • any combination of these.

Divided We Stand Political books were selected
from the New York Times Bestseller List as
starting points for 'snowball sampling'. Two
books are linked in the network if they were
purchased by the same person -- "Customers who
bought this book also bought". The pattern
reveals two distinct clusters with dense internal
ties. (early 2004)
Are these two clusters connected by non-political
books? In the map there is a path of 4 steps from
the most central Blue book to the most central
Red book. Using current fiction titles we do not
find a shorter path! Using Da Vinci Code the
centers of the clusters are 7 degrees/steps
apart, The Five People You Meet in Heaven and
South Beach Diet result is 9 degrees apart and
The Last Juror takes over 15 steps to connect the
Weaknesses in Network Analysis
  • It is difficult to get information on complete
    networks (as this involves getting information
    from all individuals/organizations). This is
    required for many of the methods involved.
  • Network analysis has been criticised for being
    better at analysing relationships between people
    (or nodes) than the structural and material
    aspects of power.

Network analysis games
  • Network analysis has been used to develop six
    degrees of Kevin Bacon (the parlour game
    developed from the notion that everyone is
    separated from everyone else by just six degrees
    of separation). The aim in Six Degrees of Kevin
    Bacon is to link any movie star to Kevin Bacon
    via films that they have both been in in less
    than six steps.
  • Can you think of anyone who is more than three
    degrees of separation from Kevin Bacon?
  • You can check your answer at http//oracleofbacon
    .org/. This site also allows you to link any
    other stars together.
About PowerShow.com