Title: Research Methods and Statistics in Psychology Lecture 12c: Qualitative Methods III
1Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
- Overview of lecture
- 1. Realist methods Content analysis
- 2. Critiques of qualitative methods
- 3. Conclusion The importance of responding to
the challenge - Reading for this lecture
- Chapter 12 in HM.
2Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
- 1. Realist methods Content analysis
- In contrast to more idealist (e.g.,
contructionist) methods, realist methods are not
dissimilar to the quantitative methods addressed
in previous chapters. - In large part this is because the method relies
on formal operations that are agreed upon by the
people who use the method and which can be
specified in courses like this one. - In the case of content analysis this quantitative
flavour is enhanced further by the fact that it
serves to transform qualitative data into
quantitative data which (if desired) can then be
subjected to formal statistical analysis.
3Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
- 1. Realist methods Content analysis
- Content analysis is a method for analysing
communication after it has been produced. - There are no boundaries to the form of
communication that can be studied indeed, the
method can be applied to the analysis of most
forms of behaviour, as most behavour has some
communicative dimension. - Importantly, though, records of the communication
must exist in some concrete form. - Despite the differences between sources of data,
there are several discrete steps that researchers
need to perform in order to analyse their content.
4Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
- 1. Realist methods Content analysis
- Step 1
- Specification of the sampling domain from which
data will be gathered - The possibilities here are infinite, but a
specific and theoretically relevant source of
data needs to be identified before research can
proceed. - As an example, let us imagine that a research
team is interested in establishing whether
managers of hospitals and managers of industrial
companies have different management styles.
Their hypothesis might be that hospital managers
have a more democratic style and that industrial
managers have a more autocratic style, but that
these differences have reduced over time.
5Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
- 1. Realist methods Content analysis
- To investigate this issue the researchers might
decide that it would be useful to examine a broad
range of managers communications their e-mails
or memos to staff, the messages they place on
noticeboards, the things they include in the
minutes of staff meetings and so on. - Whichever of these is selected, the precise
features of the domain then need to be specified
and justified on theoretical and methodological
grounds. - The considerations that guide this decision are
much the same as those that we discussed in
Lectures 4 and 5 (e.g., reducing sampling bias).
6Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
- 1. Realist methods Content analysis
- Step 2
- Development of a coding system
- As with grounded theory, this involves deciding
upon categories or coding units that can be used
to summarize key features of the data set. - In written communication, coding units are
typically specific words, phrases, or themes. In
communication more generally, they may relate to
aspects of interaction such as its duration or
intensity or the place and space in which it
occurs. - In our study of managerial style, relevant coding
units might include favourable references to
management decisions, favourable references to
employee participation and favourable
references to teamwork.
7Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
- 1. Realist methods Content analysis
- Decisions about coding categories will be devised
before data analysis begins (but usually on the
basis of preliminary examination of the material
to be coded) and be driven by both theoretical
and methodological considerations. - On the one hand, the categories need to be
relevant to the theoretical ideas being tested,
but at the same time they need to be defined at
an appropriate level of abstraction and not to
introduce systematic bias into the analysis. - For example, if our study of managerial style
included the coding unit references to share
price, it would not be surprising to find more
instances of this unit in articles about
industrial companies than in articles about
hospitals.
8Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
- 1. Realist methods Content analysis
- Step 3
- Data coding
- Here the material in the sampling domain is
systematically analysed with instances of a given
coding unit being recorded in a pre-determined
manner typically through an entry (e.g., a
tick) in the appropriate cell of a coding sheet.
- Where possible, coding is usually performed blind
to reduce the possibility of experimenter bias
(see Lecture 4). - In cases where coding systems are complex and the
content of coding categories is open to multiple
interpretations, coding will also usually be
preceded by some form of coder training. - During this, the criteria for coding decisions
are clarified and potentially difficult decisions
are discussed.
9Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
- 1. Realist methods Content analysis
- In such cases it is also customary for multiple
coders to perform the coding exercise
independently before discussing and resolving any
disagreement. - The level of pre-discussion agreement between
coders (generally the proportion of coding
choices on which they agree) can be reported as
an indication of the reliability of the coding
process. - This measure is referred to as inter-rater
reliability and, generally speaking, the higher
this is the better the coding system is seen to
have been.
10Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
- 1. Realist methods Content analysis
- Step 4
- Data summary
- Once data coding is completed, its final outcomes
can then be summarized in a range of ways. - For example, in our study of leadership style the
data might be presented in terms of the
proportion of companies in which there was at
least one instance of a given coding category, as
follows
Number of favourable references to Hospitals Hospitals Industrial companies Industrial companies
management decisions 35 65 55 50
employee participation 70 60 35 55
teamwork 15 40 25 45
11Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
- 1. Realist methods Content analysis
- Now, based on material covered in previous
lectures, we can see that the patterns here can
be described in language associated with
quantitative analysis and could be subjected to
standard forms of statistical analysis. - For example, if we look at the data in the top
row of the table, there is evidence here of a
two-way interaction between organization type and
year (a pattern discussed in Lecture10) - So the statistical significance of this pattern
could be formally established by performing a
chi-square test of independence (Lecture 11).
12Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
- 2. Critiques of qualitative methods
- From this and the previous lecture, it clear that
a broad range of options are available to collect
and analyse qualitative data. - Despite the fact that it is misleading to speak
of qualitative research as if it represented a
homogenous approach to data collection and
analysis it is still quite common for researchers
to present blanket objections to qualitative
methods on a number of grounds. - What is the basis and justification for such
criticism?
13Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
- 2. Critiques of qualitative methods
- (a) Its not scientific
- The most common criticism of qualitative methods
is that they aren't scientific. - Researchers who hold this view typically argue
that, compared to quantitative research, the
procedures for quantifying and minimizing
methodological and statistical uncertainty in
qualitative research are vague and
underdeveloped. - In the case of constructionist methods (e.g.,
discourse analysis), how can the reliability and
validity of findings be established and what
methods and criteria should be used to assess
them? - How do we know that Princess Diana's use of 'I
dunno' is a stake inoculator rather than a
statement of genuine ignorance, a habitual
response to a difficult question, or a verbal tic?
14Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
- 2. Critiques of qualitative methods
- In response to this objection, it is worth noting
first that realist qualitative methods would
allow us to answer these questions in ways that
are directly analogous to other quantitative
methods. - Such analysis might, for example, show that
Princess Diana used the phrase 'I dunno' much
more commonly after a question in which her
personal motives were questioned than when other
types of difficult question were posed. - Issues of generalization could also be addressed
through the development of appropriate theory (as
argued in Lecture 4).
15Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
- 2. Critiques of qualitative methods
- More generally, though, it is possible to argue
that the model of psychological science against
which qualitative methods are judged to be
deficient is actually misinformed. - Indeed, based on detailed analysis of the
practices in which research psychologists engage,
Woolgar suggests that psychological science (in
common with all other branches of science) is a
much less orderly and formalized activity than
idealised versions suggest. - Rather, then, than data speaking for
themselves, it appears that the form and meaning
of data is very much a product of social activity
on the part of researchers and their collective
decisions about what is important and what is
not.
16Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
- 2. Critiques of qualitative methods
- (b) Its not psychology
- A second criticism of qualitative methods is that
they don't tell us much about psychology, but are
really only useful as tools of philosophical,
sociological or linguistic analysis. - This criticism is targeted mainly at
constructionist approaches, and it is one that
many researchers in this tradition would
themselves accept. - However, such researchers would also question
whether standard quantitative approaches are
really exploring underlying psychological states
and processes in the unproblematic way that their
proponents suggest.
17Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
- 2. Critiques of qualitative methods
- (c) It falls foul of its own relativism
- A third criticism of qualitative methods is that
they fall foul of their own logic. - Again, this criticism is focused mainly on
constructionist methods. In light of the fact
that all understandings of psychological
processes are deemed to be equally valid, how
would we know whether the method had been
performed well or badly? - We alluded to this point above when noting that
discourse analysis is sometimes compared with
riding a bike something for which the
instructions cannot be spelled out but which
nevertheless people clearly succeed or fail at. - The problem here, though, is that this response
relies on a level of testable realism that the
approach itself rejects.
18Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
- 2. Critiques of qualitative methods
- (d) Its focus is too narrow
- A fourth critique is that qualitative methods
focus on the particular rather than the general
and that this undermines attempts to make
far-reaching statements about psychology and
behaviour. - This is a reasonable point, but it is possible to
make the counterargument that many important
behaviours are genuinely unique (e.g., there is
only one Diana interview) and generalization
therefore risks misinterpretation. - Against this, though, it needs to be acknowledged
that many qualitative researchers are attempting
to achieve generalization of some form. As with
quantitative research (see Lecture 4), one way
they can do this is by developing appropriate
theory.
19Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
- 2. Critiques of qualitative methods
- (e) Its for losers
- A fifth and final critique of qualitative methods
is that qualitative research is for people who
are not up to the task of conducting rigorously
designed studies and analysing complex numerical
data. - Wrong! Doing good qualitative research is every
bit as demanding as that of doing good
quantitative research. - For example, it takes approximately ten times as
long to transcribe the dialogue from an interview
as it does to conduct it, which means that a
typical study which requires this procedure
demands over 100 hours of data entry. - By comparison, the data from many experiments can
be entered and analysed in a matter of minutes. - If this is why youre attracted to qualitative
research, think again!
20Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
- 3. Conclusion The importance of responding to
the challenge - Although it is common for courses like this one
to address quantitative and qualitative research
techniques in different lectures, the above
discussion points to some of the serious problems
of this strategy. - In particular, the distinction between
quantitative and qualitative approaches is much
less clear than one might imagine. - Indeed, as we have seen, the differences between
discourse analysis and content analysis are in
many ways much more marked than those between
content analysis and standard experimental
methods. - For this reason, any approach to psychological
research which is informed by a conviction that
quantitative research is uniformly good and
qualitative research is uniformly bad (or
vice-versa) is very foolish.
21Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
- 3. Conclusion The importance of responding to
the challenge - This is not to say that a researcher should not
have a preference for one or other approach, or
that most (or all) of ones research should not
rely on a single method. - There is nothing wrong with preferring
quantitative research to qualitative (or vice
versa), or in doing research that is mainly or
exclusively quantitative (or qualitative). - However, it can be a mistake to advocate one or
other approach without considering both - (a) how your research might be enhanced by taking
an alternative approach, and - (b) how you would respond to criticism of your
research that might be levelled against it by
advocates of an alternative approach.
22Research Methods and Statistics in
PsychologyLecture 12c Qualitative Methods III
- 3. Conclusion The importance of responding to
the challenge - Science does not progress because its
practitioners cocoon themselves from ideas that
they find threatening and hard to deal with. - Instead it progresses when (and if) researchers
are prepared to rise to the challenges created by
productive intellectual tension. - Qualitative research plays an important role in
psychological science by creating tensions of
exactly this form. For this reason alone, the
challenges it lays down are well worth responding
to.