Writing up the qualitative report

1
Writing up the qualitative report assessment
criteria

• Colm Crowley
• University of Greenwich

2
Anything goes?

• Given the great diversity of structure and style
  in published qualitative research reports, how
  should we guide our students?
• While giving students a free hand can be a
  valuable part of their induction into qualitative
  methods at postgraduate level, it is more likely
  handicap them at first degree level
• For those new to qualitative methods, a
• free hand in report writing style can
• increase their bewilderment

3
Strategy - Using common principles to increase
confidence

• Build on the principles and formats common to
  both quantitative and qualitative report writing
• Highlight differences needed in qualitative
  research reports

4
Keeping it simple for 2nd year project students

• Be quite prescriptive
• Use modified scientific report format as
  framework
• But let students know that this is for pragmatic
  reasons tried and tested strategies rather than
  rigid principles
• Asking How effectively will this convey what
  readers will want to know? and How could it be
  improved further? rather than Is this right or
  wrong?

5
Modified scientific report structure

• Title
• Abstract
• Introduction
• Methodology
• Analysis (or Findings and discussion)
• Conclusion
• References
• Appendices

6
Highlighting key differences in qualitative
reports

• Differences in emphasis (compared with quant
  reports)
• -- Title
• -- Introduction
• -- Methodology
• -- Appendices
• Differences in structure (compared with quant
  reports)
• -- Findings
• -- Discussion

7
Title

• Because it still tends to be an unspoken
  assumption in much of psychology that methodology
  will be quantitative, this is not usually
  mentioned in a title (although it can perhaps be
  deduced from some titles).
• Likewise, it can be apparent from titles that
  contain words (such as narratives, accounts,
  personal meanings, lifeworlds, discourses,
  views and experiences, social/media
  representations, social construction,
  positioning, etc.) that the research reported
  will be qualitative.

8
Title

• However, it is good practice to explicitly
  indicate in a title that a qualitative
  methodology is used, as that might be of
  particular interest (or disinterest) to potential
  readers. It also makes the task of literature
  searching by methodology more efficient.
• While keeping the title as short as possible,
  inclusion of phrases such as a qualitative/ a
  phenomenological/ a grounded theory/ a
  conversation-analytic investigation/study/inquiry
  in the title (or subtitle) is regarded as
  helpful information.

9
Writing style breaking the rules?

• The writing style of a qualitative report is
  generally the same as with quantitative reports
  and other academic work (i.e., writing in an
  indirect style or reporting in the past tense).
• But writing in a more personal way, using I, is
  congruent with owning ones position
• So it is advisable to use first person forms
  where appropriate and (in student work) to state
  ones intention to do this (e.g., when discussing
  reflexivity).

10
Introduction the literature

• The literature review, while giving some
  background on the overall literature relevant to
  the topic, should ideally identify which studies
  reviewed are quantitative and which qualitative.
• When the review focuses in more depth on
  qualitative findings in the literature, it will
  assist in the subsequent discussion of the
  analysis.
• So it is worthwhile to search specifically for
  relevant qualitative literature (PsycInfos
  quant/qual methodology search filter menu can be
  useful).

11
Introduction aim

• Having summarised the rationale (based on the
  literature), the research aim can be explicitly
  stated.
• A fairly broad aim is good for an exploratory
  study. For example, to gain a greater
  understanding of the personal meanings of xxxx to
  xxxx. to gain a greater understanding of the
  lifeworlds of xxxx.
• Tip the stated aim of a piece of qualitative
  research could echo the reports title somewhat
  (in a reworded form) because it is likely that
  the scope of both (the focus and what is
  specified) would coincide.

12
Introduction research question/s

• Hypotheses usually not appropriate, certainly if
  the research aims to explore rather than verify
  (distinction between Big Q and small q
  qualitative methodology see Kidder Fine,
  1997)
• For example My initial research question was
  What are the views and experiences of the
  participants regarding xxxx? A further research
  question was What gender (cultural?) differences
  are there in the views and experiences of
  participants?

13
Useful sub-sections for Methodology

• Design
• Participants
• Methodological theory
• Method of data collection
• Procedure
• Ethical considerations
• Analytic strategy
• Reflexivity
• (but generally not Materials/ Apparatus)

14
For a conversation analysis report, Methodology
sub-sections might be

• Design
• Participants, material and context
• Procedure
• Reflexive account of selection criteria
• Transcription

15
Design sub-section

• This sub-section can be brief. The main issue is
  that of the design being flexible rather than
  fixed (see Robson, 2002) rather than just a
  qualitative design, together with which
  methodology/ies are appropriate (echoing the
  rationale in the Introduction) .
• For example, if using a single-case design
  (perhaps in an IPA study) the work of Flyvbjerg
  (2006) might be referenced.
• Similarly, if using a participant-observational
  design, the work of Flick (1998) might be
  referenced.

16
Methodological theory sub-section

• This might be GT, CA, phenomenology, or DA, for
  example. One of these fairly broad categories of
  qualitative methodology will have been mentioned
  in the Design sub-section. However, each tends to
  have some differing forms, with each form having
  its advocates and critics in the literature.
• So the task of this subsection is to identify
  which particular form of the overall methodology
  was used (and perhaps why you consider this to be
  more appropriate than alternative forms) and to
  outline its distinguishing features (supported by
  referencing)

17
Analytic strategy sub-section

• This sub-section will relate to the
  methodological theory described in the earlier
  sub-section (but be careful to avoid any
  overlap).
• It should succinctly explain the steps carried
  out in data analysis, beginning with the approach
  taken to transcription and the conventions used
  for it, e.g. as detailed by Forrester (in
  press).
• For example, themes were identified in the
  following way using procedures for IPA analysis
  described by Shaw (in press).

18
Reflexivity sub-section

• This is increasingly considered to be a criterion
  for the evaluation of qualitative research
  perhaps a qualitative parallel to quantitative
  considerations of validity and reliability, and
  seen to be a means of increasing the transparency
  of the research process.
• This sub-section should therefore demonstrate an
  understanding of the role of reflexivity in
  qualitative research (again with references such
  as Willig, 2008) and explain the procedures for
  it in the present study.

19
Reflexivity sub-section

• For example, the systematic keeping of memos
  regarding the steps of data collection and
  analysis should be reflexive and not just
  descriptive. Providing these in appendices could
  constitute an important illustration of how
  reflexivity was practiced.
• Providing a personal statement, or reflexive
  account is also often recommended, and this
  sub-section could indicate where this can be
  found in the report (most likely in an appendix
  due to word-limits).

20
Analysis

• In a qualitative research report, in order for
  the section following Methodology to make sense,
  it needs to contain a good deal of discussion.
  This is certainly the main presentational
  difference between quantitative and qualitative
  methodological reports.
• So it is more likely to be called Analysis or
  Findings (or Findings and discussion) than
  Results.
• The Analysis section will also reflect the kind
  of qualitative method used both in terms of how
  this section might be structured and how the
  content is described

21
Analysis

• For example, when presenting the quotations
  providing examples of the themes, it is necessary
  to introduce and discuss them to some extent.
• As well as this sort of contextual discussion
  (considered to be an element of the analysis in
  qualitative research), it is also necessary to
  consider the findings in the light of the
  literature reviewed in the introduction section.
• It can be more efficient to do this as you go
  along than to put it in a subsequent discussion
  section.

22
Conclusion

• However, other, more general issues need to be
  discussed by way of conclusion anyway - e.g.
  overall conclusions (interpretation of findings)
  from the Analysis section (drawing out the main
  points in the light of the research questions)
• This should relate the overall findings to the
  literature reviewed, whereas in the Analysis
  section more specific issues would have been
  discussed in the light of the literature.

23
Appendices

• Making good use of an appendix is a major element
  of a qualitative report so it can get very big!
• When students collect their own data, as with
  quantitative projects, include ethics paperwork,
  consent form, recruiting and debriefing
  information.
• When relevant to the methodology, a personal
  reflexive statement is often best included in an
  appendix (to get around word limits).
• For CA and DA, notes about or samples of specific
  material or procedures involved in the research,
  extract transcripts.

24
Appendices

• For thematic analysis (GT or IPA), at least one
  full interview transcript with (handwritten)
  initial coding on it (e.g. line-by-line coding,
  open coding).
• Also, for the thematic forms of analysis
  comprehensive lists of codes/themes/categories.
• Include all initial lists and revised lists, plus
  all other lists produced in the development of
  the analysis examples of memos examples of
  theme/category descriptions or definitions (if
  not already given in Analysis section) diagrams
  (or lists or tables) of clustered themes from
  which main themes were developed.

25
Assessment discussion points

• Comparison of marking frameworks for a
  quantitative and a qualitative 2nd year lab
  report
• Note In developing this example of a qualitative
  marking framework, the quantitative marking
  framework was adapted only where needed (rather
  than rewriting it fundamentally).
• The final (qualitative) comments box can
  highlight some limitations strengths.
• Interactive feedback? But... time issues.

26
References

• Flick, U. (1998). An introduction to qualitative
  research. London Sage.
• Flyvbjerg, B. (2006) five misunderstandings about
  case-study research. Qualitative Inquiry, 12,
  219-245.
• Kidder, L.H., Fine, M. (1997). Qualitative
  enquiry in psychology A radical tradition. In D.
  Fox I. Prillentensky (Eds.) Critical
  psychology an introduction. London Sage.
• Robson, C. (2002). Real world research (2nd ed.).
  Oxford Blackwell.
• Willig, C. (2008). Introducing qualitative
  research in psychology (2nd ed). Buckingham
  Open University Press.