Assessing Consumer Health Vocabulary Familiarity: An Exploratory Study

1
Assessing Consumer Health Vocabulary
Familiarity An Exploratory Study
Alla Keselman1,2 Tony Tse1, Jon Crowell3 Allen
Browne1 Long Ngo3 Qing Zeng3 1 US National
Library of Medicine 2 Aquilent, Inc. 3
Harvard Medical School
2
(No Transcript)
3
Study Background

• Consumers have difficulty with health texts

4
Study Background

• Consumers have difficulty with health texts
• We would like to provide support
• Authoring guidelines tools translators
• Need a way to evaluate readability
• Readability formulas
• Health domain is unique
• Familiar long words (diabetes) unfamiliar short
  words (apnea)

5
Term Familiarity Likelihood Regression Model

• Computational (regression) model
• Each term is assigned 0 1 score
• Algorithm basis
• Empirical data
• Term frequency counts from health text corpora
• Term score categories
• 0.8 1.0 score likely to be familiar
• 0.5 0.8 score somewhat likely to be
  familiar
• 0.0 0.5 score not likely to be familiar
• Source Zeng Q, Kim E, Crowell J, Tse T. A text
  corpora-based estimation of the familiarity of
  health terminology. Proc ISBMDA 2005 184-92.

6
Objectives

• Validate regression model
• Test with consumers
• Effect of demographic factors on familiarity
• Health literacy
• Education level
• Relate surface-level and conceptual familiarity
• Term vs. concept

7
Hypotheses

• Significant effect of predicted familiarity
  likelihood
• 1. Surface-level familiarity
• 2. Conceptual familiarity
• Significant effect of demographic factors
• Surface level familiarity gt conceptual

8
Survey Instrument

• 45 items hypertension, back pain, GERD
  (gastroesophageal reflux)
• Random set of terms from MedlinePlus
• Two types of test items
• Surface-level prominent association
• Surgery gt knife
• Concept level
• Surgery gt removing or repairing a body part
• 45 surface questions 15 concept questions (GERD)

9
Item Format
Modeled on the Short Assessment of Health
Literacy for Spanish-speaking Adults (SAHLSA)
Lee S-YD, Bender DE, Ruiz RE, Cho YI.
Development of an easy-to-use Spanish health
literacy test. Health Serv Res. In press.
10
Participants
11
Procedure

• Demographic survey
• Short Test of Functional Health Literacy in
  Adults (S-TOFHLA)
• Familiarity test

12
Results
Decrease
13
Results
Decrease
14
Results
15
Predictors of Surface-Level Familiarity

• Regression I
• DV surface level term familiarity
• IV Predicted Familiarity Likelihood Level,
  Gender, English proficiency, Highest Education
  Level, Age, Race, Health Literacy Level
• Significant predictors
• Predicted Familiarity Likelihood (Plt.001)
• Health Literacy (Plt.001)
• English Proficiency (P.05)

Confirms Hypothesis I
Confirms Hypothesis II
16
Predictors of GERD Concept Familiarity

• Regression II (GERD)
• DV GERD concept familiarity
• IV Predicted Familiarity Likelihood Level, GERD
  surface-level familiarity Gender, English
  proficiency, Highest Education Level, Age, Race,
  Health Literacy Level

17
Predictors of GERD Concept Familiarity

• Regression II (GERD)
• DV GERD concept familiarity
• IV Predicted Familiarity Likelihood Level, GERD
  surface-level familiarity Gender, English
  proficiency, Highest Education Level, Age, Race,
  Health Literacy Level
• Significant predictors
• Predicted Familiarity Likelihood (P.009)
• GERD surface-level familiarity score (Plt.001)
• Health Literacy (P.06) - trend

Confirms Hypothesis I
18
Predictors of GERD Concept Familiarity

• Regression II (GERD)
• DV GERD concept familiarity
• IV Predicted Familiarity Likelihood Level, GERD
  surface-level familiarity Gender, English
  proficiency, Highest Education Level, Age, Race,
  Health Literacy Level
• Significant predictors
• Predicted Familiarity Likelihood (P.009)
• GERD surface-level familiarity score (Plt.001)
• Health Literacy (P.06) - trend

Addresses Hypothesis III
19
Predictors of GERD Concept Familiarity

• Regression II (GERD)
• DV GERD concept familiarity
• IV Predicted Familiarity Likelihood Level, GERD
  surface-level familiarity Gender, English
  proficiency, Highest Education Level, Age, Race,
  Health Literacy Level
• Significant predictors
• Predicted Familiarity Likelihood (P.009)
• GERD surface-level familiarity score (Plt.001)
• Health Literacy (P.06) - trend

Trend for Hypothesis II
20
Relationship Between Surface Level and Concept
Familiarity (GERD)

• Gap between surface and concept familiarity
  (P.001)
• Size of gap greater for likely than for
  unlikely (P.006)
• Trend for somewhat likely vs. unlikely (P.07)

21
Conclusions

• Initial validity evidence for CHV familiarity
  model
• Health readability utility
• Ways to improve the model
• Allow demographic corrections
• Distinguish between knowledge of terms / concepts
• Follow-up work
• Increase sample and term pool
• Education level?
• Other predictors?
• Work on integrated findings into health
  readability formula

22
Acknowledgements

• Intramural Research Program of the US National
  Library of Medicine, US National Institutes of
  Health
• NIH grant R01 LM007222-05
• Ilyse Rosenberg for medical expertise
• Cara Hefner for help with data collection

23
Thank You!