Guidelines for Qualitative Research and Publication

1
Guidelines for Qualitative Research and
Publication

• Carolyn Seaman
• Department of Information Systems
• University of Maryland Baltimore County
• Baltimore, USA
• ISERN 2005
• 14 November 2005
• Noosa, Australia

2
What is Qualitative Research?

• Research using qualitative data
• Data in the form of text and pictures, not
  numbers or discrete categories
• Can be objective or subjective
• Tells a richer story than quantitative research
• Takes longer to tell that story
• Often used in combination with quantitative
  methods and data

3
How is qualitative research judged?

• How do we know if a piece of work is good?
• How do we do good qualitative work?
• How do we present good qualitative work
  convincingly?

4
Miles HubermanStandards for the Quality of
Conclusions

• Objectivity - treatment of contradictory evidence
  and rival explanations documentation of methods
• Reliability - competence of researchers quality
  of methods
• Internal validity - logical sense, triangulation,
  rival explanations
• External validity - representativeness
  documentation of context
• Utilization - relevance

5
Validity in Qualitative Research

• Triangulation
• Instrument validity
• Documentation

6
Triangulation

• Using a rich variety of data
• Multiple sources
• Multiple data collection techniques
• Multiple analysis techniques
• Any stated conclusion must rely on multiple types
  of evidence
• Contradictory evidence
• emerging theory must be modified in the presence
  of evidence that contradicts the current version
  of the theory
• contradictory evidence must be sought out

7
Instrument Validity

• Eliminating bias
• Do interview guides and survey questions convey
  the intended meaning? 
• Do observers understand and interpret what they
  see correctly?
• Terminology is HUGE
• Pilot, pilot, pilot

8
Documentation

• Auditing
• Traceability from data sources to collection
  methods to analysis methods to conclusions
• Bookkeeping a big job software can help
• The chain of evidence and logic that leads to any
  conclusion must be clear to any reasonable
  observer

9
Publication

• Qualitative research literature suggests various
  formats
• Telling stories
• Rich description
• Vignettes
• These wont fly in our traditional publication
  outlets

10
Publication (cont.)

• Suggested format
• Problem/motivation and related work
• Research questions
• Methods and study context
• Conclusions for each one
• Supporting evidence
• Chain of logic
• Rival explanations and contradictory evidence
• Generated theory and future work

11
Publication (cont.)

• Context use 5WH
• Who were the subjects?
• Age, gender, nationality
• Education, work experience, etc.
• What were they doing?
• Job descriptions
• Current projects
• When was data collected?
• Time of day and year
• How did it fit into their day?
• Where did data collection take place?
• Physical surroundings
• Geographical location
• Why did they participate?
• Motivations, both individual and organizational
• How was data gathered?
• Details of methods recording, format, who was
  present, etc.

12
Publication (cont.)

• Space is a BIG problem
• Write well!
• Spell out all procedures to ensure validity
  (triangulation, recording, piloting, etc.)
• Use quotes from data to illustrate conclusions
• Quantify where possible and appropriate saves
  space!