Understanding Qualitative Assessment

1
Understanding Qualitative Assessment

• Skills Session 3
• A-Team Training
• November 13, 2007

2
Session II Overview of Outcome Design
Creating Outcomes Service, Learning,
Development
Session III Critiquing Outcomes 3 Ms Intro
Assessment Steps
Session I Outcomes Overview What is
Assessment Assessment Language Comp Model
Foundation SALDOs
Session IV Assessment Steps (cont) Intro
Assessment Plans
Session V Review of Assessment Plans Session
VI Assessment Plan Presentation Session
Evaluation
Foundation Session Outcomes Demonstrate an
understanding of A language
literature Define and plan an assessment project
for your department
Increase Technical Skills Perseus, Report
Writing, Qual/Quan
Perform 1 assessment projects
Assessment Resource for department and team
3
What is Qualitative Assessment?

• Qualitative research seeks descriptively rich
  data from a small, purposeful sample with meaning
  and understanding as it end goals (Lincoln
  Guba, 1985).
• The detailed descriptions of situations, events,
  people, interactions, and observed behaviors use
  of direct quotes from people about their
  experiences, attitudes, beliefs, and thoughts
  and analysis of excerpts or entire passages from
  documents, journals, or case histories (Patton,
  1990).

4
Differences from Quantitative Assessment

• Qualitative
• -Focus of research on Quality (nature, essence)
• -Key concepts of meaning, understanding, and
  description

Quantitative -Focus of research on Quantity (how
much, how many) - Key concepts of statistical
relationships, prediction control, description,
hypothesis testing
5
Why is Qualitative Assessment important to
student affairs work?
6
Outcomes of Qualitative Assessment

• Detailed descriptions
• Direct quotations
• Analysis of documents
• Deeply rich content

7
Key Assumptions in Qualitative Assessment

• Meaning Making, understanding
• Not interested in applying broadly to situations
  outside the case being studied
• The researcher is the primary means by which data
  are collected and analyzed
• Inductive? data leads to a finding/theory
• Social reality is constructed by the participants

8
What kinds of questions would one use as a guide
for Qualitative Assessment?

• Why
• How
• Process
• Perception
• Reflection of experiences, feelings

9
Common Types of Qual Research

• Basic Interpretative Study
• Case Study
• Ethnography
• Phenomenology

10
Qualitative Techniques

• Interviews
• Focus Groups
• Document Analysis
• Observations

11
Sampling

• Key features not random, smaller than
  quantitative, provide rich information
• Several types (Patton, 2003)
• Homogeneous-common characteristics
• Stratified-sample representative of different
  characteristics of population (i.e. demographics,
  participants vs. non-participants)
• Criterion-based on predetermined criteria
• Snowball-ask participants for other potential
  participants

12
Data Analysis

• Constant Comparative Method (Glaser Strauss,
  1967)
• Constantly compare from the get-go
• Example Researcher journal-hunches, interview,
  transcribe, process is constant
• Often Data compared across multiple forms of data
  (interviews, essays, observations)

13
Coding

• Open Coding-General
• Selective Coding-Categorizing
• Thematic Coding-Themes

14
Coding Exercise

• Candy Exercise
• Interview Coding Practice

15
Coding Tips

• After interviews, write hunches down in journal
• Pay attention to what you are hearing as you
  transcribe, write down hunches
• Dont overanalyze during general coding
• Use whiteboard or flip chart to start organizing
  categories (from general codes)
• No formula, but set up in way that best suits
  your skills (Remember You are the research
  instrument ?)

16
Ensuring Rigor

• Researcher Journal
• Hunches
• Decisions
• Thoughts
• Biases, Assumptions
• Triangulation
• Data triangulation
• Researcher Triangulation
• Peer Reviewers

17
Focus Group

• Focus group-Interview a small
• group of participants (10-12 participants)
• Advantages/Disadvantages
• Permits large amount of data gathered in small
  amount of time
• Challenging to moderate multiple opinions and
  personalities
• TIP Employ another colleague to take notes!
• Read Creating a Focus Group p. 12-13 (Joels
  article).

18
Any Questions?

• For other resources, please visit our website.
• http//www.uga.edu/studentaffairs/assess/