The Validity and Reliability of Qualitative Research - PowerPoint PPT Presentation

Loading...

PPT – The Validity and Reliability of Qualitative Research PowerPoint presentation | free to view - id: 911a9-NjQ5N



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

The Validity and Reliability of Qualitative Research

Description:

An important design element, for increasing interpretive validity, therefore, is ... Codebooks (specifies definitions and relationships of concepts and terms) ... – PowerPoint PPT presentation

Number of Views:2318
Avg rating:5.0/5.0
Slides: 41
Provided by: familym
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: The Validity and Reliability of Qualitative Research


1
The Validity and Reliability of Qualitative
Research
2
Training Overview
  • Section 1 Philosophical Orientation Getting in
    the Right Mindset
  • Section 2 Enhancing the Validity of Qualitative
    Research
  • Section 3 Enhancing the Reliability of
    Qualitative Research
  • Section 4 Applications of Qualitative Research
    Design
  • Section 5 Group Work to Apply Learning

3
Section 1
  • Philosophical Orientation
  • The philosophical underpinnings of qualitative
    research (as contrasted against quantitative
    research)
  • Implications for sampling
  • Implications for addressing issues of validity
    and reliability

4
Quantitative and Qualitative Paradigms Contrasted
  • Quantitative Research is …
  • Fundamentally an inferential enterprise that
    seeks to uncover universal principles
  • Philosophically and methodologically built or
    designed around the ability to infer from a
    sample to a larger population

5
Quantitative and Qualitative Paradigms Contrasted
  • Qualitative Research is …
  • Fundamentally an interpretive enterprise that is
    context-dependent
  • Philosophically and methodologically built or
    designed around the ability to interpret
    (comprehend/understand) a phenomenon from an emic
    (insider), as well as an etic (outsider)
    perspective
  • This inter-subjective (i.e., shared)
    understanding serves as a proxy for objectivity

6
Implications for Research Questions
  • A client wants to know about how well a program
    is working for youth
  • Quantitative Research Questions (descriptive,
    explanatory -gtinferential)
  • Qualitative Research Questions
    (descriptive, explanatory -gtinterpretive)

7
Implications for Sampling
  • Sampling Strategies used in Quantitative
    Research
  • Obtaining a random or representative sample
    (based on probabilities)
  • Permits the researcher to infer from a segment of
    the population (from which it is more feasible to
    collect data) to a larger population

8
Implications for Sampling
  • Sampling Strategies used in Qualitative Research
  • Purposive sampling (to ensure that the researcher
    has adequately understood the variation in the
    phenomena of interest)
  • Theoretical sampling (to test developing ideas
    about a setting by selecting phenomena that are
    crucial to the validity of those ideas)
  • Example Case study selection in ACY (see, also,
    handout on sampling techniques)

9
Implications for Handling Threats to Validity and
Reliability
  • In quantitative research, threats to validity are
    addressed by prior design features (such as
    randomization and controls)

10
Implications for Handling Threats to Validity and
Reliability
  • In qualitative research, such prior elimination
    of threats to validity is less possible because
  • qualitative research is more inductive, and
  • it focuses primarily on understanding particulars
    rather than generalizing to universals.
  • Qualitative researchers view threats as an
    opportunity for learning
  • - e.g. researcher effects and bias are part of
    the story that is told they are not controlled
    for

11
Section 2
  • Implications
  • Enhancing the Validity
  • of Qualitative Research
  • Defining validity within the qualitative paradigm
  • Major types of validity within the qualitative
    paradigm
  • Design considerations

12
Validity
  • Validity is not a commodity that can be purchased
    with techniques … Rather, validity is, like
    integrity, character and quality, to be assessed
    relative to purposes and circumstances.

Brinberg and McGrath 198513
13
Validity
  • In general, validity concerns the degree to which
    an account is accurate or truthful
  • In qualitative research, validity concerns the
    degree to which a finding is judged to have been
    interpreted in a correct way

14
Assessing the Validity of Qualitative Research
  • Can another research read your field (and other
    types of) notes (i.e., the explication of your
    logic) and come to the same understandings of a
    given phenomenon?
  • Concern about validity (as well as reliability)
    is the primary reason thick description is an
    essential component of the qualitative research
    enterprise
  • Resources
  • Handout Different Types of Notes
  • Example ACY Site Visit Toolkit

15
Major Types of Validity in Qualitative Research
  • Descriptive Validity
  • Interpretive Validity
  • Theoretical Validity
  • External Validity (i.e., generalizability)

16
Descriptive Validity
  • Concerned with the factual accuracy of an account
    (that is, making sure one is not making up or
    distorting the things one hears and sees)
  • All subsequent types of validity are dependent on
    the existence of this fundamental aspect of
    validity

17
Descriptive Validity
  • Behavior must be attended to, and with some
    exactness, because it is through the flow of
    behavior or, more precisely, social action
    that cultural forms find articulation.

Geertz 197317
18
Interpretive Validity
  • Interpretive accounts are grounded in the
    language of the people studied and rely, as much
    as possible, on their own words and concepts
  • At issue, then, is the accuracy of the concepts
    as applied to the perspective of the individuals
    included in the account

19
Interpretive Validity Design Consideration
  • While the relevant consensus about the terms used
    in description rests in the research community,
    the relevant consensus for the terms used in
    interpretation rests, to a substantial extent, in
    the community studied
  • An important design element, for increasing
    interpretive validity, therefore, is to employee,
    at some level/to some degree, a participatory
    research approach (e.g., through member checks,
    peer to peer research model, etc.)

20
Theoretical Validity
  • Theoretical understanding goes beyond concrete
    description and interpretation its value is
    derived based on its ability to explain
    succinctly the most amount of data
  • A theory articulates/formulates a model of
    relationships as they are postulated to exist
    between salient variables or concepts
  • Theoretical validity is thus concerned, not only
    with the validity of the concepts, but also their
    postulated relationships to one another, and thus
    its goodness of fit as an explanation

21
Major Threats to Validity
  • Type I error believing a principle to be true
    when it is not (i.e., mistakenly rejecting the
    null hypothesis)
  • Type II error rejecting a principle when, in
    fact, it is true
  • Type III error asking the wrong question

22
Triangulation An Important Theoretical Validity
Check
  • Case example Parable of the blind men and the
    elephant

23
Triangulation An Important Theoretical Validity
Check
  • The most fertile search for validity comes from a
    combined series of different measures, each with
    its own idiosyncratic weaknesses, each pointed to
    a single hypothesis. When a hypothesis can
    survive the confrontation of a series of
    complementary methods of testing, it contains a
    degree of validity unattainable by one tested
    within the more constricted framework of a single
    method.

Webb et al. 1966174
24
External Validity in Qualitative Research
  • There is broad agreement that generalizability
    (in the sense of producing laws that apply
    universally) is not a useful standard or goal for
    qualitative research
  • This is not to say, however, that studies
    conducted to examine a particular phenomenon in a
    unique setting cannot contribute to the
    development of a body of knowledge accumulating
    about that particular phenomenon of interest
  • Consensus appears to be emerging that for
    qualitative researchers generalizability is best
    thought of as a matter of the fit between the
    situation studied and others to which one might
    be interested in applying the concepts and
    conclusions of that study.

25
Enhancing External Validity
  • Thick descriptions are crucial.
  • Such descriptions of both the site in which the
    studies are conducted and of the site to which
    one wishes to generalize (or apply ones
    findings) are critical in allowing one to search
    for the similarities and differences between the
    situations.
  • Analysis of these similarities and differences
    makes it possible to make a reasoned judgment
    about the extent to which we can use the findings
    from one study as a working hypothesis about what
    might occur in another situation.

26
Multi-site Studies Another Way to Enhance
Generalizability
  • A finding emerging repeatedly in the study of
    numerous sites would appear to be more likely to
    be a good working hypothesis about some as yet
    unstudied site than a finding emerging from just
    one or two sites.
  • A finding emerging from the study of several very
    heterogeneous sites would be more robust and,
    thus, more likely to be useful in understanding
    various other sites than one emerging from the
    study of several very similar sites.
  • Heterogeneity may be obtained by creating a
    sampling frame that maximizes the variation
    inherent in the sample, specifically in terms of
    potentially theoretically important dimensions

27
Section 3
  • Implications
  • Enhancing the Reliability
  • of Qualitative Research
  • Defining reliability
  • Key strategies for enhancing the reliability of
    qualitative research

28
Reliability
  • Reliability concerns the ability of different
    researchers to make the same observations of a
    given phenomenon if and when the observation is
    conducted using the same method(s) and
    procedure(s)

29
Enhancing the Reliability of Qualitative Research
  • Researchers can enhance the reliability of their
    qualitative research by
  • Standardizing data collection techniques and
    protocols
  • Again, documenting, documenting, documenting
    (e.g., time day and place observations made)
  • Inter-rater reliability (a consideration during
    the analysis phase of the research process)

30
Section 4
  • Applications of
  • Qualitative Research Design
  • Core Qualitative Methods
  • Guiding Principles
  • Qualitative Research Techniques

31
Core Qualitative Methods
  • Semi- or Un-structured, Open-Ended
  • In-depth Interviews
    (in the field, face-to-face)
  • Participant Observation
    (field/site visits)
  • Archival Research
    (document review and analysis)

32
Guiding Principles
  • Qualitative research designs consider ways to
    foster
  • Reflexivity (an ongoing process of reflecting on
    the researchers subjective experience, ways to
    broaden and enhance this source of knowing,
    examining how it informs research)
  • Iteration (a spiraling process sequential and
    repetitive steps in examining preliminary
    findings for the purposes of guiding additional
    data collection and analysis)
  • Intersubjectivity (a process of reaching a
    shared/ objective agreement about how to assign
    meaning to a social experience - with insiders
    and outsiders)

33
The Iterative Process of Qualitative Research A
Model
Analysis
Data Collection
Reflection
34
Qualitative Research Techniques
  • Instrumentation
  • Key Informants (question development and piloting
    of instrument)
  • Unstructured to Semi-structured
  • Probing
  • Data Processing and Analytic Tools

35
Qualitative Research Techniques/ Considerations
  • Sampling
  • Single v. Multiple Cases (not an individual)
  • Expert and Key Informants (identification and
    recruitment of sample)
  • Roles of the Researcher (identification and
    recruitment of sample)

36
Qualitative Research Techniques
  • Data Collection
  • Participants as Data Collectors
  • Field Notes (personal reflections, observations,
    emerging concepts/theories)
  • Debriefing (a participant, a participating
    researcher, a non-participating researcher)

37
Qualitative Research Techniques
  • Analysis
  • Key Informant Feedback
  • Codebooks (specifies definitions and
    relationships of concepts and terms)
  • Memos (emerging patterns, concepts documentation
    of analytic pathways)
  • Case Analysis Meeting (a meeting of a research
    team for the purposes of reflecting on analytic
    process, tools, and findings)
  • Matrices or Diagrams (to identify and examine
    time sequencing, the structure of relationships,
    conditions of cross case events)

38
Section 5
  • Group Work to Apply Learning

39
  • What is a given program achieving with homeless
    and runaway youth?
  • Key Methodological Issues
    (instrumentation, sampling, data
    collection, analysis)
  • What More Do You Need to Know?
  • Initial Methodological Approach and Justification

40
In-House Qualitative Resources
  • Handouts, OMNI Reports and Proposals
  • Qualitative Research Design, An Interactive
    Approach (Maxwell, 1996) Sage, Applied Social
    Research Methods Series
  • Qualitative Data Analysis (Miles Huberman,
    1994)
  • The Quality of Qualitative Research (Seale, 1999)
  • Focus Groups, Theory and Practice (Stewart
    Shamdasani, 1990) Sage, Applied Social Research
    Methods Series
  • The Mismeasure of Man (Gould)
About PowerShow.com