Research Methods and Statistics in Psychology Lecture 12c: Qualitative Methods III - PowerPoint PPT Presentation

1 / 22
About This Presentation
Title:

Research Methods and Statistics in Psychology Lecture 12c: Qualitative Methods III

Description:

In contrast to more idealist (e.g., contructionist) methods, realist methods are ... In large part this is because the method relies on formal operations that are ... – PowerPoint PPT presentation

Number of Views:131
Avg rating:3.0/5.0
Slides: 23
Provided by: alexh94
Category:

less

Transcript and Presenter's Notes

Title: Research Methods and Statistics in Psychology Lecture 12c: Qualitative Methods III


1
Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
  • Overview of lecture
  • 1. Realist methods Content analysis
  • 2. Critiques of qualitative methods
  • 3. Conclusion The importance of responding to
    the challenge
  • Reading for this lecture
  • Chapter 12 in HM.

2
Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
  • 1. Realist methods Content analysis
  • In contrast to more idealist (e.g.,
    contructionist) methods, realist methods are not
    dissimilar to the quantitative methods addressed
    in previous chapters.
  • In large part this is because the method relies
    on formal operations that are agreed upon by the
    people who use the method and which can be
    specified in courses like this one.
  • In the case of content analysis this quantitative
    flavour is enhanced further by the fact that it
    serves to transform qualitative data into
    quantitative data which (if desired) can then be
    subjected to formal statistical analysis.

3
Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
  • 1. Realist methods Content analysis
  • Content analysis is a method for analysing
    communication after it has been produced.
  • There are no boundaries to the form of
    communication that can be studied indeed, the
    method can be applied to the analysis of most
    forms of behaviour, as most behavour has some
    communicative dimension.
  • Importantly, though, records of the communication
    must exist in some concrete form.
  • Despite the differences between sources of data,
    there are several discrete steps that researchers
    need to perform in order to analyse their content.

4
Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
  • 1. Realist methods Content analysis
  • Step 1
  • Specification of the sampling domain from which
    data will be gathered
  • The possibilities here are infinite, but a
    specific and theoretically relevant source of
    data needs to be identified before research can
    proceed.
  • As an example, let us imagine that a research
    team is interested in establishing whether
    managers of hospitals and managers of industrial
    companies have different management styles.
    Their hypothesis might be that hospital managers
    have a more democratic style and that industrial
    managers have a more autocratic style, but that
    these differences have reduced over time.

5
Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
  • 1. Realist methods Content analysis
  • To investigate this issue the researchers might
    decide that it would be useful to examine a broad
    range of managers communications their e-mails
    or memos to staff, the messages they place on
    noticeboards, the things they include in the
    minutes of staff meetings and so on.
  • Whichever of these is selected, the precise
    features of the domain then need to be specified
    and justified on theoretical and methodological
    grounds.
  • The considerations that guide this decision are
    much the same as those that we discussed in
    Lectures 4 and 5 (e.g., reducing sampling bias).

6
Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
  • 1. Realist methods Content analysis
  • Step 2
  • Development of a coding system
  • As with grounded theory, this involves deciding
    upon categories or coding units that can be used
    to summarize key features of the data set.
  • In written communication, coding units are
    typically specific words, phrases, or themes. In
    communication more generally, they may relate to
    aspects of interaction such as its duration or
    intensity or the place and space in which it
    occurs.
  • In our study of managerial style, relevant coding
    units might include favourable references to
    management decisions, favourable references to
    employee participation and favourable
    references to teamwork.

7
Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
  • 1. Realist methods Content analysis
  • Decisions about coding categories will be devised
    before data analysis begins (but usually on the
    basis of preliminary examination of the material
    to be coded) and be driven by both theoretical
    and methodological considerations.
  • On the one hand, the categories need to be
    relevant to the theoretical ideas being tested,
    but at the same time they need to be defined at
    an appropriate level of abstraction and not to
    introduce systematic bias into the analysis.
  • For example, if our study of managerial style
    included the coding unit references to share
    price, it would not be surprising to find more
    instances of this unit in articles about
    industrial companies than in articles about
    hospitals.

8
Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
  • 1. Realist methods Content analysis
  • Step 3
  • Data coding
  • Here the material in the sampling domain is
    systematically analysed with instances of a given
    coding unit being recorded in a pre-determined
    manner typically through an entry (e.g., a
    tick) in the appropriate cell of a coding sheet.
  • Where possible, coding is usually performed blind
    to reduce the possibility of experimenter bias
    (see Lecture 4).
  • In cases where coding systems are complex and the
    content of coding categories is open to multiple
    interpretations, coding will also usually be
    preceded by some form of coder training.
  • During this, the criteria for coding decisions
    are clarified and potentially difficult decisions
    are discussed.

9
Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
  • 1. Realist methods Content analysis
  • In such cases it is also customary for multiple
    coders to perform the coding exercise
    independently before discussing and resolving any
    disagreement.
  • The level of pre-discussion agreement between
    coders (generally the proportion of coding
    choices on which they agree) can be reported as
    an indication of the reliability of the coding
    process.
  • This measure is referred to as inter-rater
    reliability and, generally speaking, the higher
    this is the better the coding system is seen to
    have been.

10
Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
  • 1. Realist methods Content analysis
  • Step 4
  • Data summary
  • Once data coding is completed, its final outcomes
    can then be summarized in a range of ways.
  • For example, in our study of leadership style the
    data might be presented in terms of the
    proportion of companies in which there was at
    least one instance of a given coding category, as
    follows

Number of favourable references to Hospitals Hospitals Industrial companies Industrial companies
management decisions 35 65 55 50
employee participation 70 60 35 55
teamwork 15 40 25 45
11
Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
  • 1. Realist methods Content analysis
  • Now, based on material covered in previous
    lectures, we can see that the patterns here can
    be described in language associated with
    quantitative analysis and could be subjected to
    standard forms of statistical analysis.
  • For example, if we look at the data in the top
    row of the table, there is evidence here of a
    two-way interaction between organization type and
    year (a pattern discussed in Lecture10)
  • So the statistical significance of this pattern
    could be formally established by performing a
    chi-square test of independence (Lecture 11).

12
Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
  • 2. Critiques of qualitative methods
  • From this and the previous lecture, it clear that
    a broad range of options are available to collect
    and analyse qualitative data.
  • Despite the fact that it is misleading to speak
    of qualitative research as if it represented a
    homogenous approach to data collection and
    analysis it is still quite common for researchers
    to present blanket objections to qualitative
    methods on a number of grounds.
  • What is the basis and justification for such
    criticism?

13
Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
  • 2. Critiques of qualitative methods
  • (a) Its not scientific
  • The most common criticism of qualitative methods
    is that they aren't scientific.
  • Researchers who hold this view typically argue
    that, compared to quantitative research, the
    procedures for quantifying and minimizing
    methodological and statistical uncertainty in
    qualitative research are vague and
    underdeveloped.
  • In the case of constructionist methods (e.g.,
    discourse analysis), how can the reliability and
    validity of findings be established and what
    methods and criteria should be used to assess
    them?
  • How do we know that Princess Diana's use of 'I
    dunno' is a stake inoculator rather than a
    statement of genuine ignorance, a habitual
    response to a difficult question, or a verbal tic?

14
Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
  • 2. Critiques of qualitative methods
  • In response to this objection, it is worth noting
    first that realist qualitative methods would
    allow us to answer these questions in ways that
    are directly analogous to other quantitative
    methods.
  • Such analysis might, for example, show that
    Princess Diana used the phrase 'I dunno' much
    more commonly after a question in which her
    personal motives were questioned than when other
    types of difficult question were posed.
  • Issues of generalization could also be addressed
    through the development of appropriate theory (as
    argued in Lecture 4).

15
Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
  • 2. Critiques of qualitative methods
  • More generally, though, it is possible to argue
    that the model of psychological science against
    which qualitative methods are judged to be
    deficient is actually misinformed.
  • Indeed, based on detailed analysis of the
    practices in which research psychologists engage,
    Woolgar suggests that psychological science (in
    common with all other branches of science) is a
    much less orderly and formalized activity than
    idealised versions suggest.
  • Rather, then, than data speaking for
    themselves, it appears that the form and meaning
    of data is very much a product of social activity
    on the part of researchers and their collective
    decisions about what is important and what is
    not.

16
Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
  • 2. Critiques of qualitative methods
  • (b) Its not psychology
  • A second criticism of qualitative methods is that
    they don't tell us much about psychology, but are
    really only useful as tools of philosophical,
    sociological or linguistic analysis.
  • This criticism is targeted mainly at
    constructionist approaches, and it is one that
    many researchers in this tradition would
    themselves accept.
  • However, such researchers would also question
    whether standard quantitative approaches are
    really exploring underlying psychological states
    and processes in the unproblematic way that their
    proponents suggest.

17
Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
  • 2. Critiques of qualitative methods
  • (c) It falls foul of its own relativism
  • A third criticism of qualitative methods is that
    they fall foul of their own logic.
  • Again, this criticism is focused mainly on
    constructionist methods. In light of the fact
    that all understandings of psychological
    processes are deemed to be equally valid, how
    would we know whether the method had been
    performed well or badly?
  • We alluded to this point above when noting that
    discourse analysis is sometimes compared with
    riding a bike something for which the
    instructions cannot be spelled out but which
    nevertheless people clearly succeed or fail at.
  • The problem here, though, is that this response
    relies on a level of testable realism that the
    approach itself rejects.

18
Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
  • 2. Critiques of qualitative methods
  • (d) Its focus is too narrow
  • A fourth critique is that qualitative methods
    focus on the particular rather than the general
    and that this undermines attempts to make
    far-reaching statements about psychology and
    behaviour.
  • This is a reasonable point, but it is possible to
    make the counterargument that many important
    behaviours are genuinely unique (e.g., there is
    only one Diana interview) and generalization
    therefore risks misinterpretation.
  • Against this, though, it needs to be acknowledged
    that many qualitative researchers are attempting
    to achieve generalization of some form. As with
    quantitative research (see Lecture 4), one way
    they can do this is by developing appropriate
    theory.

19
Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
  • 2. Critiques of qualitative methods
  • (e) Its for losers
  • A fifth and final critique of qualitative methods
    is that qualitative research is for people who
    are not up to the task of conducting rigorously
    designed studies and analysing complex numerical
    data.
  • Wrong! Doing good qualitative research is every
    bit as demanding as that of doing good
    quantitative research.
  • For example, it takes approximately ten times as
    long to transcribe the dialogue from an interview
    as it does to conduct it, which means that a
    typical study which requires this procedure
    demands over 100 hours of data entry.
  • By comparison, the data from many experiments can
    be entered and analysed in a matter of minutes.
  • If this is why youre attracted to qualitative
    research, think again!

20
Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
  • 3. Conclusion The importance of responding to
    the challenge
  • Although it is common for courses like this one
    to address quantitative and qualitative research
    techniques in different lectures, the above
    discussion points to some of the serious problems
    of this strategy.
  • In particular, the distinction between
    quantitative and qualitative approaches is much
    less clear than one might imagine.
  • Indeed, as we have seen, the differences between
    discourse analysis and content analysis are in
    many ways much more marked than those between
    content analysis and standard experimental
    methods.
  • For this reason, any approach to psychological
    research which is informed by a conviction that
    quantitative research is uniformly good and
    qualitative research is uniformly bad (or
    vice-versa) is very foolish.

21
Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
  • 3. Conclusion The importance of responding to
    the challenge
  • This is not to say that a researcher should not
    have a preference for one or other approach, or
    that most (or all) of ones research should not
    rely on a single method.
  • There is nothing wrong with preferring
    quantitative research to qualitative (or vice
    versa), or in doing research that is mainly or
    exclusively quantitative (or qualitative).
  • However, it can be a mistake to advocate one or
    other approach without considering both
  • (a) how your research might be enhanced by taking
    an alternative approach, and
  • (b) how you would respond to criticism of your
    research that might be levelled against it by
    advocates of an alternative approach.

22
Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
  • 3. Conclusion The importance of responding to
    the challenge
  • Science does not progress because its
    practitioners cocoon themselves from ideas that
    they find threatening and hard to deal with.
  • Instead it progresses when (and if) researchers
    are prepared to rise to the challenges created by
    productive intellectual tension.
  • Qualitative research plays an important role in
    psychological science by creating tensions of
    exactly this form. For this reason alone, the
    challenges it lays down are well worth responding
    to.
Write a Comment
User Comments (0)
About PowerShow.com