Module 4: Appraisal, Extraction and Pooling of Evidence arising out of qualitative research

1
Module 4 Appraisal, Extraction and Pooling of
Evidence arising out of qualitative research
2
Qualitative Research is

• a situated activity that locates the observer
  in the world. It considers a set of interpretive,
  material practices that make the world
  visibleit involves an interpretive,
  naturalistic approach to the world. This means
  qualitative researchers study things in their
  natural settings, attempting to make sense of, or
  to interpret, phenomena in terms of the meanings
  people bring to them
• (Denzin Lincoln, 2000)

3
Qualitative research and health care

• Health care is a people centred process concerned
  with health and healing
• Health care derives much of its knowledge from a
  range of biomedical sciences
• However this does not supply all the knowledge
  that is necessary to provide holistic care for
  the patients/clients
• Qualitative researchers use humanistic frameworks
  to examine ordinary activities of everyday life

4
Qualitative Methodologies
5
Qualitative Research Findings as Evidence for
Practice

• Qualitative evidence is of increasing importance
  in health services policy, planning and delivery.
• It can play a significant role in
• understanding how individuals and communities
  perceive health, manage their own health and make
  decisions related to health service usage
• increasing our understandings of the culture of
  communities and of health units in relation to
  implementing change and overcoming barriers to
  the use of new knowledge and techniques
• Informing planners and providers in relation to
  how service users experience health and illness
  and the health system
• evaluating components and activities of health
  services that cannot be measured in quantitative
  outcomes (such as health promotion and community
  development).

6
Application of Qualitative research to practice

• As with quantitative research, results from a
  single study only should not be used to guide
  practice
• The findings of qualitative research should be
  synthesised in order to develop recommendations
  for practice

7

• Researchers and policy makers are increasingly
  recognising that findings from qualitative
  studies aiming to explore the experience of those
  involved in providing and receiving interventions
  as well as studies using multiple methods to
  evaluate the factors that shape the process of
  implementing interventions, have an important
  role in ensuring that systematic reviews are of
  maximum value to policy, practice and consumer
  decision-making

8
The Debate...

• The synthesis or pooling of the findings of
  qualitative research studies is controversial.
• Contested by quantitative researchers because of
  the subjective nature of qualitative evidence.
• Contested by qualitative researchers because of
  the ideological, philosophical and methodological
  differences that characterise the flexibility of
  the qualitative research tradition.
• Some qualitative researchers argue that the
  synthesis of qualitative studies is impossible
  and meaningless. Others support the notion of
  qualitative synthesis, but there is no emerging
  consensus on appropriate guidance for the
  systematic review of qualitative evidence for
  health and social care.

9
Integration - Aggregation versus Interpretation

• The two dominant, opposing views that
  characterise the ongoing debate surrounding the
  meta-synthesis of qualitative evidence focus on
  integration or aggregation versus interpretation.

10
Integration/aggregation

• Involves assembling the findings of studies
  (variously reported as themes, metaphors,
  categories etc) and pooling the findings through
  further aggregation based on similarity in
  meaning.
• Those who oppose this approach suggest that
  synthesis represents new knowledge and that
  aggregation is not synthesis.
• Integrative syntheses are those where the focus
  is on summarising data, and where the concepts
  (or variables) under which those data are to be
  summarised are assumed to be largely secure and
  well specified
• The kinds of results that integrative syntheses
  may be especially likely to produce will often be
  theories of causality, and may also include
  claims about generalisability

11
Interpretive Synthesis

• involves both induction and interpretation, and
  are concerned not to predict but to anticipate
  what might be involved in analogous situations
  and to understand how things connect and
  interact.

12
Differing Approaches
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
Guidance on the synthesis of qualitative findings

• The Cochrane Qualitative Research Methods Group
  has not yet reached consensus on the preferred
  approach for Cochrane Reviews
• JBI has adopted an integrative/aggregative
  approach for JBI reviews

17
The JBI Approach

• JBI recognises epistemological and practical
  differences between research traditions but
  concur with the Health Development Agency (HDA
  2004) who state that
• ... much of the methodological debate is
  completely unhelpful... ... does nothing to help
  develop policy and practice... and .. is a
  gross oversimplification of vast amounts of
  important scientific work in a range of
  methodological traditions...
• Estabrooks et al (1994) make a case for
  undertaking what they call aggregation of
  qualitative studies, arguing that analysis and
  synthesis of multiple studies can result in a
  contribution to theory-building more powerful
  than any single study. More pragmatically,
  failing to optimise the outcomes of the
  time-consuming investment in qualitative research
  is clearly wasteful (Thorne, 1994).

18
Meta-Synthesis...

• A neologism
• Meta
• ... Beyond transcending more comprehensive.
• Synthesis
• ... the combining of the constituent elements of
  separate material or abstract entities into a
  single or unified entity.
• ...a complex whole formed by combining.
• ...the combination of components to form a
  connected whole.
• ...The combination of separate elements of
  thought into a whole, as of simple into complex
  conceptions, species into genera, individual
  propositions into systems the opposite of
  analysis.

19

• Synthesis is defined as the process of combining
  different ideas, influences or objects into a new
  whole.
• The qualitative researcher, in analysing
  qualitative data, is involved in synthesising the
  words of participants or the text of observation
  notes through categorising and developing
  descriptors in the form of themes or other
  groupings.
• Meta synthesis, as a higher order form of
  synthesis, is a process of combining the findings
  of individual studies (that is, cases) to create
  summary statements that authentically describe
  the meaning of these themes (or cross-case
  generalisations).
• Meta-synthesis is an interpretive process but
  requires transparency of process.

20
The Process of Metasynthesis

• Attempts to mirror the Cochrane process modelled
  on an integrated, thematic analysis process
• The JBI approach to the metasynthesis of the
  findings of qualitative research studies is
  embodied in the Qualitative Assessment and Review
  Instrument (QARI) QARI (pronounced Quarry) helps
  reviewers to mine or dig for evidence.

21
The Qualitative Assessment and Review Instrument
(QARI)
22
QARI was designed to provide a systematic process
mirroring that taken for systematic reviews of
quantitative research whilst being sensitive to
the nature of qualitative data.
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
Critical Appraisal of Evidence arising out of
qualitative research
28
(No Transcript)
29
Critical Appraisal of Interpretive Research

• This focus on limiting bias to establish validity
  in the appraisal of quantitative studies is
  antithetical to the philosophical foundations of
  qualitative approaches to inquiry
• In interpretive and critical inquiry validity
  relates to the rigour of the process of inquiry
• QARI incorporates a checklist to appraise rigour
  generically

30
(No Transcript)
31
(No Transcript)
32

• Group Work
• Critical Appraisal of Evidence arising out of
  qualitative research
• Report Back

33
Introduction to Data Extraction and
Meta-synthesis
34
Data Extraction
35
Aims

• Reduce
• The findings of many studies into a single
  document
• Summarise
• Methods
• Phenomena
• Findings

36
(No Transcript)
37

• a significant proportion of qualitative work is
  not designed to address questions or issues of an
  immediate practical nature
• this is consistent with the traditions from which
  many qualitative methodologies are derived
• SR can only include reports that can be regarded
  as offering valid and useful evidence for practice

38
Extracting Findings

• The units of extraction in this process are
  specific findings and illustrations from the text
  that demonstrate the origins of the findings
• In QARI a finding is defined as A conclusion
  reached by the researcher(s) and often presented
  as themes or metaphors

39
(No Transcript)
40
Assigning a Level of Credibility to Qualitative
Evidence
41

• Levels of Credibility- Qualitative Studies
• Unequivocal - relates to evidence beyond
  reasonable doubt which may include findings that
  are matter of fact, directly reported/observed
  and not open to challenge
• Credible - those that are, albeit
  interpretations, plausible in light of data and
  theoretical framework. They can be logically
  inferred from the data. Because the findings are
  interpretive they can be challenged.
• Not Supported - when 1 nor 2 apply and when most
  notably findings are not supported by the data

42
(No Transcript)
43
Meta-Synthesis
44

• the analysis and synthesis of qualitative
  studies is commonly termed meta-synthesis, and
  like meta-analysis, it is based on processed
  data.
• The aim of meta-synthesis is to assemble
  findings categorise these findings into groups
  on the basis of similarity in meaning and to
  aggregate these to generate a set of statements
  that adequately represent that aggregation. These
  statements are referred to as synthesised
  findings - and they can be used as a basis for
  evidence based practice.

45
Data Synthesis Involves

• Step 1 Identifying findings
• Step 2 Grouping findings into categories and
• Step 3 Grouping categories into synthesised
  findings

46
When Engaging in the Synthesis of the Results of
Qualitative Studies, Note That

• Differing research methods, such as
  phenomenology, ethnography or grounded theory,
  can be mixed in a single synthesis of qualitative
  studies because the synthesis is of findings and
  not data. This is a critical assumption of the
  QARI process. QARI meta-synthesis does not
  involve a reconsideration and synthesis of
  primary data-it is restricted to the combination
  of findings. Contrary to Noblit and Hares view,
  we consider it unnecessary to restrict
  meta-synthesis to studies undertaken using the
  same methodology

47

• As qualitative studies differ from RCT,
  meta-synthesis also utilises an approach that is
  markedly different from that used during
  meta-analysis. At the conclusion of both these
  approaches, the product of the synthesis provides
  an understanding that is based on a range of
  populations, settings and circumstances. This
  broad base for generation of evidence on a
  phenomenon allows for greater confidence in the
  evidence. However, unlike meta-analysis,
  meta-synthesis deals in multiple realities and so
  provides but one interpretation of the phenomenon.

48
Executing meta-synthesis using QARI
49
(No Transcript)
50
(No Transcript)
51
(No Transcript)
52
(No Transcript)
53
(No Transcript)
54
(No Transcript)
55
(No Transcript)
56
(No Transcript)
57
Meta-Synthesis
58
Synthesised Findings

• In QARI, a synthesised finding is defined as
  an overarching description of a group of
  categorised findings that allow for the
  generation of recommendations for practice.

59
Formulating synthesised findings

• Can be stated propositionally as if-then
  statements - for example If a patient is
  awaiting a final diagnosis, their relatives will
  sometimes feel as if they are not involved. (a
  somewhat awkward and eccentric form)
• We prefer the declamatory form that emphasises
  the probability of the claim Relatives of
  patients awaiting a final diagnosis of brain
  death may feel as if they are not involved if
  strategies to include them are not pursued.

60
Executing Meta Synthesis using QARI
61
(No Transcript)
62
(No Transcript)
63
(No Transcript)
64
(No Transcript)
65
(No Transcript)
66
QARI TrialReport Back
67
Further Protocol Development