Addressing Stress and Addictive Behavior in the Natural Environment Using AutoSense

1
Addressing Stress and Addictive Behavior in the
Natural Environment Using AutoSense

• Santosh Kumar
• Computer Science, University of Memphis

2
Our Team

• Behavioral Science

• Engineering

• Dr. Mustafa alAbsi, UMN
• Dr. J Gayle Beck, Memphis
• Dr. David Epstein, NIDA, NIH
• Dr. Tom Kamarck, Pittsburgh
• Dr. Satish Kedia, Memphis
• Dr. Kenzie Preston, NIDA, NIH
• Dr. Marcia Scott, NIAAA, NIH
• Dr. Saul Shiffman, Pittsburgh
• Dr. Annie Umbricht, Johns Hopkins
• Dr. Kenneth Ward, Memphis
• Dr. Larry Wittmers, UMN

• Dr. Anind Dey, CMU
• Dr. Emre Ertin, Ohio State
• Dr. Deepak Ganesan, UMass
• Dr. Greg Pottie, UCLA
• Dr. Justin Romberg, Georgia Tech
• Dr. Dan Siewiorek, CMU
• Dr. Asim Smailagic, CMU
• Dr. Mani Srivastava, UCLA
• Dr. Linda Tempelman, Giner Inc.
• Dr. Jun Xu, Georgia Tech

3
Students Postdocs

• Memphis

• CMU, OSU, UCLA, Georgia Tech., UMN

• Dr. Andrew Raij (now at USF)
• Dr. Kurt Plarre
• Dr. Karen Hovsepian
• Amin Ahsan Ali
• Santanu Guha
• Monowar Hussain
• Somnath Mitra
• Mahbub Rahman
• Sudip Vhaduri

• Dr. Motohiro Nakajima, UMN
• Patrick Blitz, CMU
• Brian French, CMU
• Scott Frisk, CMU
• Nan Hua, Georgia Tech
• Taewoo Kwon, OSU
• Moaj Mustang, UMass
• Siddharth Shah, OSU
• Nathan Stohs, OSU

4
Paradigm Shift in Disease Prevalence

• Infectious diseases, and those from poor hygiene
  nutrition not as prevalent
• They are replaced by diseases of slow
  accumulation
• Heart diseases
• Cancer, Ulcer
• Depression, Migraine

5
Growing Epidemic Stress Addiction

• Stress addictive behavior lead to or worsen
  diseases of slow accumulation
• Stress headaches, fatigue, heart failures,
  hypertension, depression, addiction, anxiety,
  rage
• Smoking cancer, lung diseases, heart diseases
• Yet, both continue to be widespread
• Stress 43 adults suffer adverse health effects
• Smoking responsible for 20 of deaths in US
• An urgency to help individuals reduce
  stress abstain from addictive behavior

Stress costs 300 billion/yr
Smoking costs 193 billion/yr
6
Addressing Stress Addiction

• An unobtrusively wearable sensor suite called
  AutoSense
• So, individuals can wear it in natural
  environment
• Robust inference of stress from physiological
  measures
• Automatically measure physiological and
  psychological stress
• Automatic inference of addictive behaviors
• Smoking, drinking, drug usage from sensor
  measurements
• Detect addiction urges to provide timely
  intervention
• Craving for smoking and drug usage
• Contexts/cues that may lead to craving and
  eventual relapse
• Infer other moderating behavioral social
  contexts
• Conversation, physical activity, traffic
  stressors, etc.

7
Outline

• Hardware and Software Platforms
• AutoSense sensor suite
• FieldStream mobile phone framework
• Inferring Stress
• Detecting stress from physiology
• Predicting perceived stress
• Ongoing User Studies
• Detecting smoking, drinking, craving, drug usage,
  etc.
• Roadmap Long-term Vision

8
AutoSense Wearable Sensor Suite
Android G1 Smart Phone
9
Key Features of AutoSense Hardware

• Ultra low power
• Six sensors (ECG, GSR, Resp., Temp, Accel)
  consume 1.75 mA
• Overall current consumption lt 3mA (for 10 days
  of lifetime)
• Sampling and transmission of 132 samples/sec
  (i.e., 1.8 kbps)
• Reliable radio
• ANT with integrated quality of service and duty
  cycling
• Reliable and timely wireless transmissions in
  crowding scenarios
• Antenna impedance is matched for human body
• Power loss reduced from 33 (for free space
  configuration) to 0.1
• Operates at 2480-2524 MHZ band to be immune to
  Wi-Fi
• Average packet loss rate of 0.57 even when Wi-Fi
  activity is intense

10
FieldStream Mobile Phone Framework

• For use in conducting scientific user studies
• In both supervised lab settings and in
  uncontrolled field settings
• It collects measurements
• Sensor measurements from wearable and phone
  sensors
• Self-reports from subjects
• Computes tens of features and various statistics
  over them (e.g., HR, HRV, RR, Minute Ventilation)
• Makes inferences using machine learning
  algorithms
• Stress, posture, activity, conversation, and
  commuting
• Detects sensor detachments and loosening
• Is reconfigurable
• So, no need for change in source code for use in
  a new user study

11
Computes base features (e.g., R-R interval)
statistics over them
Provides a common interface to all sensors
populates buffers for feature computation
Converts stream of sensor measurements into
packets delivers to intended recipient
12
Deployment Experiences and Findings

• 21 subjects in UMN - completed
• Lab session on stress 10-14 hours per day for 2
  days in field
• 36 subjects in Memphis - completed
• 3 consecutive days in field with daily visits to
  the lab
• Some findings on human behaviors in our subject
  pool
• Stress occurrence in daily life (Plarre et. al.,
  in ACM IPSN11)
• Subjects were psychologically stressed 26-28 of
  time
• Natural conversations (Rahman et. al., in ACM
  Wireless Health11)
• Frequency of conversations 3 per hour
• Avg. duration of a conversation 3.82 minutes
• Avg. Time between conversations 13.3 minutes

13
Outline

• Hardware and Software Platforms
• AutoSense sensor suite
• FieldStream mobile phone framework
• Inferring Stress
• Detecting stress from physiology
• Predicting perceived stress
• Ongoing User Studies
• Detecting smoking, drinking, craving, drug usage,
  etc.
• Roadmap Long-term Vision

14
Measuring Stress in the Field

• Self-reports have been used for a long time
• Questionnaires or surveys
• Measures perceived stress
• Strengths and limitations
• () Captures detailed information
• () Proximal predictor of mental health
• (-) Distal predictor of physical health
• (-) Discrete sampling
• (-) Burden to participant
• Need an automated approach for continuous stress
  measurement in the field

15
Continuous Measure of Stress

• Can use physiological measurements to assess
  stress, but
• Physiology is affected by several factors, not
  only stress

• Activity, change in posture, speaking, food,
  caffeine, drink, etc.
• How to separate out the changes in physiology due
  to stress?

• How to map physiology to psychology?

16
The Quest for Automated Stress Measure

• Predicting psychological state from physiology
• William James pioneering work (1880)
• John Cacioppo and others revitalized interest
  (1990)
• Several studies on emotion and stress prediction
• Identified physiological markers of stress and
  emotion Example Heart rate, skin conductance
  response
• But, confined to controlled settings
• Few studies in uncontrolled environments
• M. Myrtek96 , J. Healey05, J. Healey10,
• Either no validated stressors, no lab session to
  train models, not able to account for
  confounders, or tried to match self-reports
  directly

17
In the AutoSense Project

• We developed a new wearable sensor suite
• Conducted a scientific study with validated
  stress protocol
• 21 participants, 2 hour lab study, 2 day field
  study
• Protocol designed by behavioral scientists
• Stressors used are validated and known to produce
  stress
• Self-reports designed by expert behavioral
  scientists
• Developed new stress models to measure
• Physiological response to stress
• To measure adverse physiological effects of
  stress
• Perception of stress in mind
• To derive a continuous rating of perceived stress

18
Lab Study Stress Protocol

• 2 hour lab session
• Subjects exposed to three types of stressors
• Public speaking psychosocial stress
• Mental arithmetic mental load
• Cold pressor physical stress
• Physiological signals recorded at all times
• Using AutoSense
• Also, collected self-reported stress rating 14
  times

Public Speaking
Cold Pressor
Mental Arithmetic
Baseline
Recovery
Start
End
10 Min
10
10
4
4
4
4
4
4
4
10
10
10
4
5
5
5
19
Self-Report Measures of Stress

• Self-report questions related to affective state

Question Possible Answer Code
Cheerful? YES yes no NO 3 2 1 0
Happy? YES yes no NO 3 2 1 0
Frustrated/Angry? YES yes no NO 0 1 2 3
Nervous/Stressed? YES yes no NO 0 1 2 3
Sad? YES yes no NO 0 1 2 3
20
(No Transcript)
21
Our Aproach
22
Identified 22 Features from Respiration
Basic Features
Statistical Features
Inhalation Duration
Mean
Exhalation Duration
Median
Respiration Duration
80th Percentile
Insp./Exp. Ratio
Quartile Deviation
Stretch
Breathing Rate
Minute Ventilation
23
Computed 13 Features from ECG
Basic Features
Statistical Features
Variance
Power in low/medium/high frequency bands
RR Intervals
Ratio of low frequency/high power
Mean
Median
RSA
80th Percentile
Quartile Deviation
24
Feature and Classifier Selection

• Used Weka for Training
• Evaluated Decision Tree, DT with Adaboost, and
  Support Vector Machine
• Using 10-fold cross validation, and training/test
  data
• Classification results using 35 features
• After feature selection, 13 features
• 8 Respiration, 5 ECG

J48 Decision Tree J48 with Adaboost SVM
87.67 90.17 89.17
25
Classification Accuracy on Lab Data
26
Our Aproach
27
Perceived Stress Model

• Use a binary Hidden Markov Model
• To reduce number of parameters, we approximate
  by
• ? models the gradual decay of stress with time
• ? models the accumulation of stress in mind due
  to repeated exposures to stress
• Both ? and ? are person dependent and are learned
  from self-reported ratings of stress

28
Evaluation of the Model (on Lab Data)

• Correlation of perceived stress model and
  self-report rating in the lab session
• Over 21 participants
• Median correlation
• 0.72

29
Field Study Protocol

• Participants wore AutoSense continuously for 2
  days
• Going about their daily life (home, school, etc.)
• Except when sleeping at night
• Field self-reports
• Participants responded to self-reports 20 times
  each day
• Same questions about affect state as in the lab
• Additional context information
• Additional behaviors automatically collected
• Speaking, from respiration patterns
• Physical activity, from accelerometer

30
Realities of Natural Environment

• Data eliminated
• 37 affected by activity
• 30 by bad quality
• Less than 4 min consecutive data
• 4 subjects missing data or self-report

31
Evaluation of the Model (Field)

• Compared average stress ratings over both days
• Accumulation model versus self-report
• Linear interpolation

32
Outline

• Hardware and Software Platforms
• AutoSense sensor suite
• FieldStream mobile phone framework
• Inferring Stress
• Detecting stress from physiology
• Predicting perceived stress
• Ongoing User Studies
• Detecting smoking, drinking, craving, drug usage,
  etc.
• Roadmap Long-term Vision

33
Ongoing User Studies

• Memphis Study
• 40 daily smokers and social drinkers
• A lab study followed by one week in the field
• Stress, drinking, smoking, and craving for
  cigarettes marked
• National Institute on Drug Abuse (NIDA) Study
• 20 drug addicts undergoing treatment
• Two lab sessions and 4 weeks in field
• Smoking, craving, and stress marked in lab
• Craving, stress, and drug usage reported in the
  field
• Johns Hopkins Study
• 10 drug addicts in residential treatment
• Drug injection in lab, daily behaviors marked in
  the field
• To develop detectors for smoking, craving, and
  drug usage

34
Roadmap

• The near-term goal is to develop personalized
  stress and addiction assistants on the mobile
  phone to
• Help reduce stress, e.g., least stressful route
  for driving
• Break addiction urges where and when they occur
• But, these applications will impact someones
  health
• Will it indeed be helpful to each user and not
  hurt anyone?
• Will it help maintain healthy behaviors even
  after the novelty phase?
• How do we generate evidence for its validity,
  efficacy, safety?
• Within reasonable time and effort, unlike
  multiyear RCTs
• How do we design it so it has greater chance of
  success?
• Various theories exist (e.g., stages of change,
  social cognitive theory)
• But, no overall theory for designing adaptive
  interventions exist today

35
Long-term Vision

• Use these experiences to discover the scientific
  principles that can be used broadly in mobile
  health (mHealth)
• To design and develop
• New mHealth measures that are robust enough for
  field usage
• New mHealth treatments and interventions that
  work
• To generate evidence of validity, efficacy, and
  safety of mHealth

Contribute to the newly emerging science of
mHealth
36
Further Reading

1. E. Ertin, N. Stohs, S. Kumar, A. Raij, M.
   al'Absi, T.Kwon, S. Mitra, Siddharth Shah, and J.
   W. Jeong, AutoSense Unobtrusively Wearable
   Sensor Suite for Inferencing of Onset, Causality,
   and Consequences of Stress in the Field, ACM
   SenSys, 2011.
2. Md. Mahbubur Rahman, Amin Ahsan Ali, Kurt Plarre,
   Mustafa al'Absi, Emre Ertin, and Santosh Kumar,
   mConverse Inferring Conversation Episodes from
   Respiratory Measurements Collected in the Field,
   ACM Wireless Health, 2011.
3. Mohamed Mustang, Andrew Raij, Deepak Ganesan,
   Santosh Kumar and Saul Shiffman, Exploring
   Micro-Incentive Strategies for Participant
   Compensation in High Burden Studies, to appear
   in ACM UbiComp, 2011.
4. K. Plarre, A. Raij, M. Hossain, A. Ali, M.
   Nakajima, M. al'Absi, E. Ertin, T. Kamarck, S.
   Kumar, M. Scott, D. Siewiorek, A. Smailagic, and
   L. Wittmers, Continuous Inference of
   Psychological Stress from Sensory Measurements
   Collected in the Natural Environment, ACM IPSN,
   2011.
5. Andrew Raij, Animikh Ghosh, Santosh Kumar and
   Mani Srivastava, Privacy Risks Emerging from the
   Adoption of Inoccuous Wearable Sensors in the
   Mobile Environment, In ACM CHI, 2011.

Nominated for best paper award
Nominated for best paper award
37
Outline

• Hardware and Software Platforms
• AutoSense sensor suite
• mStress mobile phone framework
• Inferring Stress
• Detecting stress from physiology
• Predicting perceived stress
• Ongoing User Studies
• Detecting smoking, drinking, craving, drug usage,
  etc.
• Privacy Issues in mHealth research

38
Behavior Revelation from Sensors

• Accelerometer gyroscopes can be used to monitor
  activity level
• Can infer movement pattern and place from these
  sensors
• See SenSys10 paper on AutoWitness
• Could also infer epileptic seizures
• Respiration sensor can be used for activity
  monitoring or estimating the extent of pollution
  exposure
• Can use it to infer conversation, smoking, and
  stress
• Inferring of public speaking episodes could even
  pinpoint the identity of the subject
• Development of other behavioral inferences in
  progress

39
How Concerned are Study Participants?

• Conducted a 66 subject (36 in NS) study
• Evaluated their concern level as their personal
  stake in the data is increased
• Also, how their concern level changes as
  modalities are added/removed

40
Awareness Concern

• Sharing of stress, commuting, and conversation
  generate higher concern than the sharing of place

41
Effect of Privacy Transformations

• Disassociating time is more critical than
  disassociating place of occurrance
• Even reducing timestamp to duration helps