QUALITATIVE METHODS: AN OVERVIEW

1
QUALITATIVE METHODS AN OVERVIEW

• SHOSHANNA SOFAER, DR.P.H.
• SCHOOL OF PUBLIC AFFAIRS
• BARUCH COLLEGE

2
SUMMARY OF TOPICS

• How is qualitative work different
• When qualitative work is most useful
• Sample selection
• Data collection methods
• Data management and analysis

3
KEY FEATURES

• Focus on the case rather than on the variable
• Focus on the development of understanding and
  meaning rather than the testing of hypotheses
• Focus on the context of events/activities as well
  as the events/activities themselves
• Recognize the multiplicity of perspectives that
  can be brought to any event/activity

4
KEY FEATURES

• Myths about qualitative research (as compared to
  quantitative studies)
• It is not objective
• It is not rigorous
• It is not, therefore, trustworthy
• It is easy anyone can do it

5
KEY FEATURES

• Realities of qualitative research
• Behind every quantity there must lie a quality
  if you dont understand the quality on its own
  terms, you cannot quantify it in a valid manner
• Subjectivity is an element of all research
  claiming objectivity and lack of bias is at best
  disingenuous and at worst dangerously dishonest

6
KEY FEATURES

• Realities of qualitative research
• The potential for sloppy thinking and execution
  is present for all research some qualitative
  research is sloppy but the best is highly
  rigorous
• The data collection, interpretation and analysis
  methods of this kind of research are probably
  more difficult the researcher is a critical
  tool

7
KEY FEATURES

• Adding rigor to qualitative research
• Make assumptions explicit and question them
• Separate statements that describe what has been
  observed from analyses/interpretations of what
  has been observed
• Triangulate use multiple observers multiple
  respondents multiple methods
• Look for evidence to disconfirm emerging
  conclusions

8
WHEN IS IT USEFUL?

• When there is little prior empirical research on
  a topic
• When prior research
• Results in conflicting or inconclusive findings
• Leaves key processes and assumptions unexamined
• Leaves out the perspective of key stakeholders,
  including the less powerful

9
WHEN IS IT USEFUL?

• When you have some idea what your question is but
  you dont know enough to have hypotheses
• When you have no idea of the range of responses
  you might have to your questions
• When relevant events are changing rapidly
• When the context seems critical

10
WHEN ISNT IT USEFUL?

• When you can predict and specify the range of
  responses to your questions
• When you are testing a causal hypothesis
• When your primary audience will only have
  confidence in quantitative findings

11
WHEN IS IT USEFUL?

• Examples from my own recent experience
• Focus groups with recent hospital patients to get
  their reactions to a set of nursing sensitive
  quality measures (NQF15)
• Key informant interviews with hospice providers
  to explore their use of quality measures and
  their response to potential public reporting of
  such measures
• Cognitive testing of materials to report on
  Hospital CAHPS

12
SAMPLE SELECTION

• A key feature of qualitative research is
  purposive rather than random sampling
• Of research sites (cases)
• Of key informants
• Of focus group participants
• Of events to observe
• Of documents to analyze

13
SAMPLE SELECTION

• There are exceptions, however, when you do want
  to move toward a more representative sample
• For example, when you have a large population of
  highly similar units (sites, informants, events,
  documents) to choose from

14
SAMPLE SELECTION

• What is purposive sampling?
• Conscious selection by the researcher based on a
  set of prior concerns
• Examples include
• Choosing extremes on important dimensions
• Choosing prototypes
• Choosing informants or sites with specific
  experiences or knowledge

15
SAMPLE SELECTION

• Issues in sample selection
• Getting a sampling frame or population of
  possibilities from which to choose
• Learning enough about the units in the sampling
  frame to make an appropriate choice
• Getting detailed information to make contact
• Getting access/informed consent
• Maintaining access

16
SAMPLE SELECTION

• Notice that these issues are extremely similar to
  those faced, for example, in a survey research
  study
• A critical issue do your homework learn as
  much as you can BEFORE you begin collecting data
  about people, context, etc.

17
DATA COLLECTION

• Major Methods
• Interviews
• Observations
• Focus Groups
• Document or Object Analysis

18
INTERVIEWS

• Key dimensions of interviews
• How structured?
• How administered?
• How frequent?
• How many respondents per interview/overall?
• How many interviewers per interview/overall?

19
INTERVIEWS

• How Structured? The options are
• Unstructured
• Semi-structured
• Structured

20
INTERVIEWS

• Unstructured interviews are used in highly
  exploratory, ethnographic studies
• They involve little more than the generation of a
  set of general topics and the initiation (by a
  skilled interviewer) of a conversation on those
  topics
• Examples

21
INTERVIEWS

• Semi-structured interviews are probably the most
  common and most useful approach in HS/HP research
• Involve use of a written protocol consisting of
  OPEN ENDED questions, sometimes with suggested
  probes
• Exact wording, question sequence and use of
  probes at discretion of the interviewer

22
INTERVIEWS

• Key skills for conducting semi-structured
  interviews
• Building rapport and either authority or
  sympathy with respondent
• Observing and using non-verbal cues (when doing
  interviews in person)
• Knowing when to probe when to drop questions
  when to let the person talk when/how to refocus
  them

23
INTERVIEWS

• Asking open- v. closed-ended questions
• Did you like participating in the Penn State
  Qualitative Methods Workshop?
• What aspects of the Penn State Qualitative
  Methods Workshop were most valuable to you? What
  aspects could be significantly improved?

24
INTERVIEWS

• Asking open- v. closed-ended questions
• Do you think your Aligning Forces for Quality
  partnership has the right mix of members?
• What if any organization or constituency is not
  included in your AF4Q partnership who really
  needs to be there?

25
INTERVIEWS

• Structured interviews are extremely close to a
  survey, although they still use primarily
  open-ended questions
• Wording, sequence and probes are pre-determined,
  not at discretion of interviewer
• This means that a less experienced (but still
  well trained) interviewer can be used

26
INTERVIEWS

• How administered? The options are
• In person
• Telephone
• NOT MAIL ALONE although a list of topics can be
  provided ahead of time to the respondent(s)

27
INTERVIEWS

• Issues to consider
• Geography
• Number of respondents
• Cost
• Expected difficulties establishing rapport
• Sensitivity of the topics covered in the protocol
• Ability to record data easily

28
INTERVIEWS

• Interview frequency depends on the nature of your
  research design, e.g. whether you are looking for
  changes over time
• Once rapport is created through an initial
  in-person contact, you can use phone interviews
  for follow-up interviews

29
INTERVIEWS

• How many respondents in a given interview?
• Usually one, especially in phone interviews, but
  can interview 2-5 people in person
• Issues include
• Hierarchy among respondents
• Limits on candor taking the party line
• Dealing with the quiet respondent

30
INTERVIEWS

• How many interviewers?
• In telephone interviews, almost always one
• In-person, best to have two interviewers to
• Provide reality test/debriefing
• Deal with recording, including non-verbal info
• Create training opportunity

31
INTERVIEWS

• When its better to just use one person
• When the topic is sensitive
• When the respondent is vulnerable or wary
• When resource limits wont let you do enough
  double up interviews

32
FOCUS GROUPS

• Grows out of marketing research
• Increasingly used in health services and health
  policy research
• Basic structure is a one to two hour session with
  a carefully chosen group of people who discuss a
  series of specific focus questions under the
  guidance of a skilled moderator

33
FOCUS GROUPS

• Why do we conduct focus groups?
• To explore an issue about which we are developing
  a survey instrument
• To identify the shared and dissimilar concerns of
  specific sub-groups of people
• To test messages we want to use in communication
  efforts

34
FOCUS GROUPS

• Who typically participates in focus groups?
• Community members
• Clients/patients
• Staff members
• Students
• In some cases, policy-makers

35
FOCUS GROUPS

• Why a focus group rather than a series of
  interviews?
• Primary reason is that we believe we can learn
  something from the way the people in the group
  react to each other as well as to the questions
  from the focus group moderator

36
FOCUS GROUPS

• Key issues in focus groups
• What are your inclusion and exclusion criteria
  (overall, for each group)?
• How will you recruit participants?
• Will you use incentive payments?
• How many groups will you run?
• How many people in each group?

37
FOCUS GROUPS

• Key issues in focus groups (continued)
• What are your focus questions?
• Are you using/reviewing/sharing materials?
• Will you also collect closed-ended data (e.g.
  demographics, specific opinions)?
• How will you record the information?
• Do you need bi-lingual and/or bi-cultural
  moderators?

38
DOCUMENT/OBJECT ANALYSIS

• Health care organizations and health policy
  making processes produce reams of documents, in
  print and now on websites
• Other objects such as video or audio-tapes,
  photographs, paintings and other works of art,
  are also created in the course of delivering care
  or making policy

39
DOCUMENT/OBJECT ANALYSIS

• All these materials can be subjected to analysis
  in much the same way that you will later analyze
  interview tapes, notes or transcripts, field
  notes of observations, or video tapes of focus
  groups

40
DOCUMENT/OBJECT ANALYSIS

• Particularly in policy research, pioneering work
  is beginning in using the method of discourse
  analysis to examine, for example, the
  transcripts of Congressional hearings or floor
  debates

41
DATA ANALYSIS

• The data in qualitative analyses are either
  text or images or a combination
• As in quantitative research, these data must be
  carefully collected and managed so that
  researchers know what data have been collected,
  when and by whom, what has been transcribed,
  coded, recoded, etc.
• Make copies of everything!!!

42
DATA ANALYSIS

• Most rigorous qualitative data analysis involves
  the search for patterns and themes in the data,
  and an exploration of the context in which
  certain patterns, themes and language occur
• The goal is to find meaning where possible, but
  also to recognize when reality remains
  imponderable

43
DATA ANALYSIS

• Some forms of qualitative data collection lend
  themselves to quantification, such as
  post-coding and counting of responses or
  behaviors, especially when you use highly
  structured interviews or observations
• That approach, however, is an exception to the
  rule

44
DATA ANALYSIS

• Coding is at the heart of qualitative data
  analysis -- this is also called content
  analysis and overlaps with document and image
  analysis
• Codes are typically brief descriptive phrases
  that are assigned to specific pieces of text or
  to images

45
DATA ANALYSIS

• Steps in coding
• Data immersion reading all or a large
  proportion of the transcripts or notes
• Development of a preliminary coding scheme with
  1-3 levels of code
• Coding (by multiple trained analysts where
  possible)
• Refinement of coding scheme and recoding

46
DATA ANALYSIS

• Computer software is now available to assist in
  the data analysis process
• This helps when data are computerized it takes
  the tedium out of coding and permits the
  researcher to see information both in and out of
  its textual or visual context
• The primary value of software, in my view, is the
  ability to pull together all text elements with
  the same codes, or that have two codes assigned

47
DATA ANALYSIS

• As with quantitative analysis, this is an
  iterative process
• Interpretation is both essential and dangerous
• Look for alternative plausible explanations and
  test the data for them

48
DATA ANALYSIS

• It really really helps to be working with
  colleagues who can and will keep you honest by
  insisting that you make the path to your
  explanation explicit
• Nevertheless, it remains true that experience and
  skill have an impact on the subtlety of
  interpretations
• A key skill is keeping the context in view

49
FINAL COMMENT

• Health services and policy research are moving
  more in the direction of qualitatative and mixed
  methods
• We should only use these methods when we can use
  them appropriately and carry them out rigorously
  as well as creatively
• Good luck!