Judging Qualitative Research - PowerPoint PPT Presentation


PPT – Judging Qualitative Research PowerPoint presentation | free to download - id: 610cb6-YjI2O


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

Judging Qualitative Research


Judging Qualitative Research The Role of the Reader – PowerPoint PPT presentation

Number of Views:45
Avg rating:3.0/5.0
Slides: 32
Provided by: quig1
Learn more at: http://psychology.tcd.ie


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Judging Qualitative Research

Judging Qualitative Research
The Role of the Reader
  • "There are no operationally defined truth tests
    to apply to qualitative research" (Eisner, 1991,
    p. 53).
  • Researcher and readers "share a joint
    responsibility" for establishing the value of the
    qualitative research product (Glaser and Strauss,
    1967, p. 232).
  • "Pragmatic validation of qualitative research
    means that the perspective presented is judged by
    its relevance to and use by those to whom it is
    presented their perspective and actions joined
    to the researchers perspective and actions"
    (Patton, 1990, p. 485).

  • Validity of research corresponds to degree to
    which it is accepted as sound, legitimate and
    authoritative by people with an interest in
    research findings.
  • How do we judge which perspective to use to
    evaluate the validity of a qualitative study?
  • E.g., grounded theory should theoretically sample
    a wide range of people discourse analysis can be
    in-depth analysis of a few excerpts

  • Process of agreeing criteria for judging
    qualitative research is useful because involves
    critically reflecting on essential ingredients
  • But simply following guidelines does not
    guarantee good research not simply a
    descriptive science but relies on capacity to
    evoke imaginative experience reveal new

3 features
  • Coherence (structural corroboration or
  • Consensus
  • Instrumental Utility
  • "Guides call our attention to aspects of the
    situation or place we might otherwise miss"
    (Eisner, 1991, p. 59

  • "How can an inquirer persuade his or her
    audiences that the research findings of an
    inquiry are worth paying attention to?" (Lincoln
    and Guba 1985, p. 290).
  • Lincoln and Guba (1985, p. 300) - an alternative
    set of criteria that correspond to those
    typically employed to judge quantitative work.

Comparison of criteria
  • Conventional terms Naturalistic terms
  • Internal validity Credibility
  • External validity Transferability
  • Reliability Dependability
  • Objectivity Confirmability

Critical of use of "comparable criteria
  • little different than the conventional criteria
  • assumes that what is known existent or
    interpreted reality stands independent of the
    inquirer and can be described without distortion
    by the inquirer
  • naturalistic research can offer only an
    interpretation of the interpretations of others
  • to assume an independent reality is unacceptable
    for many qualitative researchers

e.g., Smith Heshusius 1986
  • there is no "out there" out there the only
    reality is a completely mind-dependent reality,
    which will vary from individual to individual
  • therefore, not possible to choose a best
    interpretation from among the many available,
    because no technique or interpretation can be
    "epistemologically privileged" (p. 9).

  • This stance prohibits the possibility of
    reconciling alternative interpretations
  • Important to determine which criteria are
    consistent with the naturalistic paradigm, yet
    which allow for a declaration that good research
    has been carried out.
  • Select appropriate criteria for judging overall
    trustworthiness of a qualitative study

Internal Validity vs. Credibility
Conventional inquiry Naturalistic inquiry
Extent to which findings accurately describe reality. Must postulate relationships and then test them. Postulate cannot be proved, only falsified. Assume the presence of multiple realities and try to represent these multiple realities adequately. Isomorphism is in principle impossible if we knew precise nature of reality, no need to test it. Credibility becomes the test for this.
Credibility-A toolbox of procedures for
enhancing validity
  • Triangulation Enrich understanding of a
    phenomenon by viewing from different
  • 4 types
  • methods triangulation
  • data triangulation
  • triangulation through multiple analysts
  • theory triangulation.
  • gather data from different groups of people
    gather data at different times from same people
    different theories/methods composite analysis
    triangulate perspectives of different

  • Comparing researchers coding
  • Inter-rater comparison of subtle, complex coding
    schemes participant feedback/respondent
  • Making segments of the raw data available for
    others to analyze, and the use of "member
    checks," in which respondents are asked to
    corroborate findings.

  • Disconfirming case analysis
  • Deviant or negative cases, systematically
    searching for data that does not fit themes
  • Audit (paper) trail
  • Provide evidence linking raw data to final report

Audit trail
  • Consisting of
  • raw data
  • analysis notes
  • reconstruction and synthesis products
  • process notes
  • personal notes and
  • preliminary developmental information
  • Critical component conceptual chain of logic
    - mimics replicability process in conventional

External Validity / Generalizability versus
Conventional seek findings which are statistically generalisable Naturalistic aspire to theoretical, vertical or logical generalization of findings
Ability to generalize findings across different settings. Involves a trade-off between internal external validity (can incl. only limited aspects of each local context). transferability of a working hypothesis to other situations depends on the degree of similarity between the original situation and the situation to which it is transferred. Existence of local conditions makes it impossible to generalize.
Reliability versus Dependability
Conventional Inquiry Naturalistic Inquiry (Focus instead on achieving greater validity in work)
1) the degree to which a measurement, given repeatedly, remains the same 2) the stability of a measurement over time 3) the similarity of measurements within a given time period use an inquiry audit in which reviewers examine both the process and the product of the research for consistency. "Since there can be no validity without reliability ( thus no credibility without dependability), a demonstration of the former is sufficient to establish the latter" (Lincoln Guba 1985 316).
Objectivity versus Confirmability
Conventional Inquiry Naturalistic Inquiry
Relies on quantitative measures to define a situation is relatively value-free, therefore objective. Subjectivity leads to unreliable invalid results. Relies on interpretations and is admittedly value-bound, is therefore considered to be subjective. Question the possibility of ever attaining pure objectivity.
Empathic Neutrality
  • Patton (1990) the terms objectivity and
    subjectivity have become "ideological ammunition
    in the paradigms debate." Use instead "empathic
    neutrality" (p. 55).
  • Empathy "is a stance toward the people one
    encounters, while neutrality is a stance toward
    the findings" (p. 58). A researcher who is
    neutral tries to be non-judgmental, and strives
    to report what is found in a balanced way.
  • Lincoln and Guba (1985) the "confirmability" of
    the research - the degree to which the researcher
    can demonstrate the neutrality of the research
    interpretations, through a "confirmability

Demonstrating validity
  • Sensitivity to context allow patterns
    meanings NOT prespecified to emerge
  • Relevant theoretical empirical literature
  • Socio-cultural setting
  • Participants perspectives
  • Ethical issues
  • Empirical data

Sensitivity to the context of existing theory research in the development of a research topic Study lacks features of validity Study sets out to simply explore womens experiences of postnatal depression, ignoring relevant theory previous qualitative studies of experiences of maternity depression in women Study demonstrates features of validity Study clarifies what is already known form theory research, formulates a specific research question that has not been addressed How does the family context influence womens experiences following childbirth?
Sensitivity to how the perspective position of participants may influence whether they feel able to take part express themselves freely Senior male professional carries out interviews with women with postnatal depression in a clinic setting, ignoring possibility they may feel less able to express feelings to a man, my be intimidated by clinical environment. Participants give choice of taking part in focus groups (allowing solidarity with pother women in similar experiences) or interviews in own home (privacy, security, accessibility) carried out by women of own age.
  • Commitment rigour
  • through data collection
  • Depth/breadth of analysis
  • methodological competence/skill
  • In-depth engagement with topic

Commitment rigour in recruitment of participants who will represent an adequate range of views relevant to research topic Sample comprises 12 self-selected volunteers. Most are well-educated, affluent, married. 12 women are purposively sampled to include married, co-habiting single participants from affluent and socio-economically disadvantaged backgrounds.
  • Coherence transparency
  • clarity power of argument
  • fit between theory and method
  • transparent methods data presentation
  • Reflexivity

Transparency in the analysis of data Little description is provided of how themes were identified and no checks on their consistency reported A detailed description is provided of how data were initially coded and how codes were modified through comparison of all instances and discussions between the researchers.
Coherence between the qualitative design the analysis and presentation of data Based on a frequency count of the occurrence of codes, the researcher highlights the finding that 2/3 of the single women (n3) but only 1/3 of married women complain of lack of social support. Strong quantitative statements of this kind are inappropriate given small sample size and reliability of codes is unknown. Based on a qualitative comparisons of all instances of the codes, the researchers note that there was a tendency for single women to report a lack of social support. However disconfirming instances are discussed as revealing examples of why married cohabiting women may feel unsupported and how single mothers may be supported by others.
  • Impact and Importance
  • practical/applied
  • theoretical
  • socio-cultural

Impact of the research The researcher simply notes that the findings are compatible with existing models and research (e.g., showing that single mothers feel they have less social support). The researchers explain how different family structures and relationships may exert positive or negative influences on the experience of maternity, suggesting questions for further research and ways of identifying and supporting women at risk of depression.
  • Not always possible to meet some criteria,
    sometimes must prioritize some kinds of validity
    over others
  • A set of principles to refer to when making
    decisions about how to carry out to justify
    your research.

  • Willig Chapter 9.
  • Barbour, R.S. (2001) Checklists for improving
    rigour in qualitative research a case of the
    tail wagging the dog? British Medical Journal,
  • Cho, J. Trent, A. (2006) Validity in
    qualitative research revisited. Qualitative
    Research, 6, 319-340.
  • Harré, R. (2004) Staking our claim for
    qualitative psychology as science. Qualitative
    Research in Psychology, 1, 3-14.
  • Henwood, K. (2004) Reinventing validity. In
    Todd, Z., Nerlich, B., McKeown, S. Clarke, D.D.
    (Eds.). Mixing methods in Psychology The
    Integration of Qualitative and Quantitative
    Methods in Theory and Practice. Psychology
    Press. Chapter 3.
  • Parker, I. (2005) Qualitative Psychology
    Introducing Radical Research. OUP. Chapter 10.
About PowerShow.com