Designing Risk Communications (implications from Comprehension Tests)

1
Designing Risk Communications (implications from
Comprehension Tests)

• Louis A. Morris, Ph.D.
• Drug Information Association
• June 15, 2004

2
Objectives

• Review Information Processing Models
• Patients (novices)
• Physicians (experts)
• Describe Models for Risk Communications
• Pharmacokinetics of Information Processing
• Discuss Learnings from Comprehension Tests
• Cognitive Load Problems
• Signaling Problems
• Discuss Role of Readability Comprehension
  Testing

General Conclusions, Not Specific Findings
3
Information Processing
Information Processing/ Memory
Perceptual Buffer Attention Processes
Stimuli/ Messages
Environmental Cues/Usage Needs
Decisions
4
Implications

• What captures attention has best opportunity to
  be processed
• Placement top of document, beginning of
  sentences
• Graphics Headers, bold, italics
• Webers Law
• Limited Capacity
• We remember what is important --- depends on how
  we will use data

5
Cognitive Processes

• Decoding and interpreting words
• Lexical Processing
• Extracting meaning within and among sentences
• Syntactical Processing
• Discerning overall theme
• Discourse Processing
• Meaning is Constructed
• Like a house is built
• More resources needed to build foundation, less
  available for other sections

Constructing a Schema
6
Tell Me What this Says?

• The black and white cow walked over to the
  purple esuoh and smiled broadly to give the namow
  a liap of milk that she made into a nollag of
  French vanilla ice cream to sell to the srotisiv.

7
Implications

• Simple words
• Short Sentences
• However
• Short words may not be simple
• Concept is difficult and may need explanation
• Some large words help organization
• Sing-Song sentences are a turn-off

Readability Formula
8
(No Transcript)
9
Types of Processing

• Top-Down experts (physicians)
• Start with Existing schema and modify
• More efficient preferred mode of processing
• Information stresses differences
• Bottom-Up novices (patients)
• Provide building blocks, simplify information
• Reduce cognitive load, provide signals
• Provide intellectual scaffold (organization)
  for new information

10
Principle of Cognitive Conservation

• People have limited cognitive resources
• Limited Capacity Working Memory
• We can think about 7 /- 2 bits
• Distribute Cognitive Resources"
• Based on our goals
• What is the Cognitive Load
• How much information to process
• How difficult is the information to process

Interaction of Materials load and structure
with Patients Information Processing Skills and
Motivations
11
Pharmacokinetic Communications Model
Motivation
Involvement Goals
Cognitive Load
Willingness to Process
Situational Constraints
Opportunity
Patient
Document
Signals
Literacy Self-Efficacy
Ability
Actual Processing
Morris Aiken
12
Ability

• Literacy
• NALS five literacy levels, 40 of US _at_ 1 or 2
• Level 1
• 25 immigrants
• 33 elderly (65 yrs. or older)
• 25 physical or mental problem
• Self-efficacy
• If people do not expect to be able to perform,
  they wont try
• Self-efficacy judgments may not be based on
  sufficient information

13
Risk Communication Options

• HCPs
• PI, Label Changes (black box), Dear Doctor
  letters, Advertisements (affirmative disclosure,
  separate campaign)
• Patients
• PPI, Medication Guides, Informed Consent,
  Agreement
• Educational Campaigns
• Public (PR)
• FDA public announcements (talk papers, press
  releases), website posting, advisory committee
  meetings

Vary in Format and Cognitive Load
14
Comprehension Tests

• Started with OTC Drugs
• Advil/Nuprin LL vs. CS Label
• Nicotine Products
• Common for Switches
• Applied to Medication Guides
• Other risk management documents
• Applied to Physician Labels

15
General Procedure

• Recruit (n 400 to 1,200)
• Use Shopping Malls
• Screen for at-risk population
• Disease characteristics
• Low Literacy (pronunciation tests)
• Design
• One Cell Survey
• Multi-Cell Comparisons

16
General Procedure (2)

• Procedure
• Screening
• Document Exposure read as normally would
• Interviewer Leaves Room
• Questionnaire
• Develop Communication Objectives
• Funnel Approach
• Open ends
• Specific Communication Objectives
• Follow-up Questions
• Document usually present (may be taken away for
  initial open ends)

17
MGs vs. OTC Labels

• Longer
• Cognitive constraints on information processing
• Limited take away, time to interpret
• More difficult words/concepts
• Needs explanation to understand consequences
• More Complex Directions
• Application to variety of usage situations
• Risk Topics
• warnings, contraindications, side effects list
• Associated Documents
• Agreements, wallet cards, consent, audiovisual

18
Comprehension Test Findings

• Cognitive Load - Longer documents
• Primary Points fully communicated (extensive
  repetition) (over 90 correct)
• Secondary Points poorly communicated (66-75
  correct)
• Evidence Participants adopt a harm prevention
  bias consult physician in response to any
  issue, regardless on information in the material.
  
• Do not attempt to consult document to answer
  questions
• Questions requiring multiple mental operations
  are very difficult (more than three items to
  consider leads to very poor results)

Tradeoffs- Explanation vs. Length
19
Lessons from Comp. Tests (1)

• Simplification
• Avoid Extensive Repetition
• Use brief headers in most important section
• Avoid Seductive Details
• Interesting to know information detracts from
  key messages
• Use Communications Objectives to focus messages
• Cut out extra words

20
Lessons form Comp. Tests (2)

• Signals
• Less likely to notice information in
• second half of a long sentence,
• middle of a paragraph,
• not graphically emphasized
• Graphic emphasis works surprisingly well for OTC,
  not for MGs
• too much background noise reduce number of
  sub-heads
• Simplifying language and concepts
• very often necessary but often requires much
  background

21
Lessons from Comp. Tests (3)

• Context Matters
• Vague words decrease comprehension
• do not drink alcohol
• Health, sickness, etc., has unclear meaning
• Credibility and Persuasion Needed
• To influence behavior, need to go beyond
  comprehension
• Rationale for advocated behaviors may be needed

22
Is Readability Testing Enough?

• Can help simplify information
• Does not address overall length and cognitive
  burdens due to overall flow
• Does not address simple words that are difficult
  to interpret (red meat, healthy, sick)
• Question of Predictive Validity
• Do readability tests accurately predict reading
  level
• Tests developed in late 1940s

23
(No Transcript)
24
Conclusions

• MGs (and other documents) are here
• FDA required for drugs with risk management
  problems.
• Affirmative Approach (ie, write your own)
• Bring FDA own designed/tested
• Readability can help simplify, Comprehension
  Testing is defensible
• Comprehension tests for liability defense
• Plan Testing Prior to Implementation
• Changing document after use can imply weaknesses
  in prior document