When Users Pledge to Take Green Actions, Are They Solving a Decision Problem?

1
When Users Pledge to Take Green Actions, Are They
Solving a Decision Problem?
Michael Johnson University of Massachusetts Boston Boston MA 02125 michael.johnson_at_umb.edu Susan Fussell (sfussell_at_andrew.cmu.edu) Jennifer Mankoff (jmankoff_at_cs.cmu.edu) Deanna Matthews (dh5x_at_andrew.cmu.edu) Leslie Setlock (lsetlock_at_andrew.cmu.edu) Carnegie Mellon University Pittsburgh PA 15213
INFORMS Fall Conference, Washington, D.C. October
15, 2008
2
Acknowledgements

• Funding
• Intel Corporation, Leveraging Computational
  Technologies to Support Behavior Change,
  Jennifer Mankoff, principal investigator
• National Science Foundation grants NSF
  IIS-0745885 and NSF IIS-0205644, Jennifer
  Mankoff, principal investigator
• Research assistance
• Victoria Yew

3
Problem Motivation

• Excessive energy consumption is a primary cause
  of global warming
• Americans consumed 100 quadrillion BTUs of energy
  (U.S. Department of Energy 2006)
• Energy consumption primarily linked to individual
  activities lighting, heating and cooling (Bin
  and Dowlatabadi 2005, U.S. Environmental
  Protection Agency 2006)
• Numerous Web-based initiatives exist to encourage
  environmentally responsive behavior
• Daily actions GreenSpeak.org
• Green travel EarthRoutes.net
• Carbon footprint calculations and green living
  Yahoo!Green
• Social networking sites enable individuals to
  meet, collaborate and act collectively
• Facebook
• MySpace

4
Research questions

• Can we design an application to encourage
  reductions in daily activities that affect global
  warming that integrate
• On-line social networking, to attract people who
  are not necessarily green
• Displaying progress towards energy reduction
  goals, individually and in groups
• Tracking actual energy consumption through
  sensors
• Multiple technology platforms
• Result is StepGreen, an initiative that combines
  scholarship, outreach and social change
• Mankoff, Matthews, Fussell and Johnson (2007)
• Mankoff et al. (2008)

5
StepGreen A Web-based system in support
individual action to combat global warming
StepGreen.com actions for commitment
Social network site badge
Cumulative effects of actions taken
Source www.stepgreen.org
6
Alternative disciplinary views of StepGreen

• Technology a flexible, robust system will induce
  behavioral change and attract many users
• IS/IT policy a popular on-line application will
  provide novel insight into IT adoption, usage and
  outcomes.
• Decision sciences a Web-based decision support
  system will help ordinary users make better and
  more efficient decisions about high-impact daily
  actions
• Public policy learn about long-term social and
  environmental impacts of individual change as
  compared to national-level policy.

7
Action selection is a decision problem

• Behavioral change is a multi-step process
• Persuade users that action on a particular topic
  is urgent
• Learn consequences of actions along multiple
  dimensions
• Explore alternatives by examining tradeoffs in
  attribute space
• Choose one or more actions that optimize utility
• Observe actual impacts of actions
• Update preferences for action attributes

8
Action selection is a decision problem, contd

• Action representation is important to
  decisionmaking
• Present choices in lists of varying sizes
• One at a time
• List
• Present choices in differing ways
• Text descriptions
• Tables, charts and graphs
• Dynamics in action representations
• Static
• Interactive

What is the effect of length and information
content of alternative visual representations of
actions on decision time and decision quality?
9
Theoretical Foundations

• Human-computer interactions (Dix, et al. 2004)
• Role of visual representation in decision-making
• Lurie and Mason (2007)
• Miller (2004)
• Mandel and Johnson (2002)
• Decision aids for large/complex decision problems
• Payne et al. (1988)
• Eiselt and Sandblom (2004)
• Decision support systems for consumer choice
• Häubl and Trifts (2000)
• van der Heijden (2006)
• Decision-making styles and barriers
• Bruine de Bruin, Parker and Fischoff (2007)
• Scott and Bruce (1995)

10
Research gap decision aids and DSS for
public-sector problems

• Limited literature on decision support and
  visualization especially by unsophisticated
  users, or those in vulnerable or underrepresented
  groups (Johnson 2006)

Can specific decision aids and visualization
strategies enable users to make decisions
regarding lifestyle choices more effectively, or
with higher levels of satisfaction?
11
Previous experiment design

• Surveys
• Effects of human actions on global warming
• Decision-making styles
• Satisfaction with range of choices provided and
  rankings made
• Evaluate text-based actions according to
• Length (terse vs. verbose)
• Information content (relevant vs. irrelevant)
• Action category (e.g. Heating, Lighting,
  Appliances, Water)
• Actions are partitioned into two sets of
  non-dominated alternatives
• Superior (8 or 10)
• Inferior (2 of 10)
• Users rank top four actions out of 10 available
  in four categories

12
Previous experiment text representation
Water Consumption
13
Previous experiment results

• Decision quality shows significant variation with
  action categories
• Few statistically significant associations with
  decision length or quality
• Question length
• Information content
• Decision style
• Satisfaction with the range of decision
  alternatives
• Satisfaction with the ranking decisions
• No evidence of learning about environmental
  impacts

14
Will alternative visualization methods make a
difference?

• Hypothesis users prefer graphical
  representations of actions to text
  representations and will make better decisions.
• New design
• Four action categories
• Four visual representations of action
  characteristics and impacts
• Terse/relevant text
• Symbols
• Value path
• Bar chart

15
Experiment data classifications, values

• Actions and categories
• Actions came from literature review and common
  sense
• Categories inspired by card-sorting exercises
• Appliances
• Heating/Cooling
• Lighting and Appliances
• Water Consumption
• Impacts
• Carbon emissions
• Change in energy usage Energy Star
  (http//www.energystar.gov/)
• Estimates of carbon savings Energy Information
  Administration (http//www.eia.doe.gov/emeu/aer/tx
  t/ptb1207b.html)
• Dollar costs/savings Energy Information
  Administration (http//www.eia.doe.gov/emeu/aer/tx
  t/ptb0810.html)
• Time costs/savings rules of thumb
• Quality of life subjective assessments

16
Actions Text (Appliances)
17
Actions Symbols
18
Actions Value path
19
Actions Bar chart
20
Research framework, contd

• Users are assumed to act according to defined
  decision-making styles (Scott and Bruce 1995)
• Intuitive
• When I make a decision I trust my inner feelings
  and reactions
• Rational
• I make decisions in a logical and systematic
  way
• Dependent
• I often need the assistance of other people when
  making important decisions
• Avoidant
• I often procrastinate when it comes to making
  important decisions
• Spontaneous
• When making decisions I do what seems natural at
  the moment
• Users learn about environmental impacts of
  various actions through choice process

21
Hypotheses

• H 1 Participants decision quality and decision
  time vary according to action category.
• H 2a Graphical representation results in better
  outcomes than text representations
• H2b Decision time and decision quality varies
  according to specific graphical representations
• H 3a Rational decision-making styles are
  associated with higher-quality decisions
• H 3b Spontaneous decision-making styles are
  associated with lower-quality decisions
• H 3c Rational decision-making styles are
  associated with slower decisions
• H 3d Intuitive decision-making styles are
  associated with more rapid decisions
• H 4 Participants showed an increase in knowledge
  about impacts of specific actions with respect to
  global warming
• H 5 Gender is associated with decision quality
  and decision time

22
Experiment design

• Within-subjects design four conditions (Martin
  2004)
• Survey software automatically randomized
  presentation orders and recorded decision times
• Action categories, graphical representations
  counterbalanced across participants
• 32 undergraduate and graduate student
  participants
• Steps
• Study overview and consent forms
• Pretest survey
• Action choices
• Posttest surveys
• Compensation

23
Results Descriptive Statistics

• Participant characteristics
• 72 male
• 91 students (60 undergraduates 31 graduate
  students)
• 84.4 between 18 25 years old
• Most live in households with unrelated mates and
  no children
• Diverse racial, ethnic backgrounds
• Decision-making styles (means of 1 to 5
  scaled question responses within categories)
• Rational 3.82
• Intuitive 3.59
• Dependent 3.26
• Avoidant 2.68
• Spontaneous 2.87

24
Results Descriptive Statistics, contd

• Baseline global warming knowledge is generally low

25
Results Outcome measures

• Dominated responses number of choices that were
  dominated in the subjects responses 0, 1, 2
• Response time time to make all selections used
  log-transformed times due to skewed original
  values

26
Results Effect of action category

• Mixed model analyses showed no difference between
  domains (appliances, heating, lighting, water)
  for
• Number of dominated choices (F 3, 36.3 1.71,
  p .17) or
• Log response time (F 3, 58.93 1.13, p .35)

No effect on decision quality or time due to
action category
27
Result Effect of representation type

• Collapsed outcomes over action categories, giving
  each participant one score for each visualization
  condition
• Dominated choices
• Log response time

Representation type has no effect on decision
quality Response times longer for text than for
graphics, and response times do not differ by
graphic type
28
Results Pre- and post-test learning
After experiment, users generally perceived
greater global warming impact on all actions
29
Correlations user characteristics and decision
outcomes

• Mean number of dominated choices per trial
  negatively correlated with log of response time
  ( -0.405 0.021)
• Gender (male 0, women 1) is negatively
  correlated with log of response time (-0.439
  0.012)
• Decision-making style has no statistically
  significant effect on decision-making quality or
  decision time
• Gender not statistically significantly correlated
  with decision style or decision quality

30
Correlations graphical representations and
outcomes

• Some relationships between decision times across
  graphical representations
• Text and value path (.470 0.008)
• Text and bar chart (.513 0.010)
• Symbols and value path (.603 0.000)
• Symbols and bar chart (.403 0.051)

31
So, are StepGreen users solving a decision
problem?

• We think so..but decision context so far provides
  little support.
• For static representations of actions
• Generally, neither length of action descriptions,
  information content within descriptions or
  graphical representation of actions had
  significant effects on decision outcomes.
• Why did more information about decision
  alternatives not help subjects make better
  decisions?
• Alternatives ranking too demanding cognitively?
• Information insufficiently tailored to different
  needs, interests and backgrounds of subjects?

32
Implications for research

• Design
• For maximum speed, emphasize graphical
  representations of actions
• Particular graphical representation not important
  
• Next steps
• Use a human intermediary to help users choose
  actions
• Apply new methods to measure decision-making
  competence

33
New Experiment Human-assisted decisionmaking

• Inspiration
• Risk-perception literature (Florig, et al. 2001)
  detailed problem representation improves risk
  ranking
• Decision competence literature (Parker, Bruine de
  Bruin and Fischoff 2007), in which measures of
  decision efficacy are associated with
  satisfaction with decisions made
• Idea
• Use a human intermediary to help users better
  understand their own values and preferences and
  characteristics of action alternatives

34
Goal evaluate decision outcomes along two
dimensions of interventions

• Intermediary types
• Peer intermediary - informal, youthful affect
  and use a minimum of technical language
• Expert intermediary - more formal, academic
  affect, use technical language and appear to be
  an authority on behavioral changes and impacts of
  actions on climate change.
• Intermediation type
• Quantitative - Problem-focused, scientific
  presentation of the impacts of various actions
  using figures and descriptions of relevant
  calculations
• Qualitative - individual-focused, interactive,
  holistic discussion using probing questions to
  learn about subject attitudes regarding different
  actions

35
Scientific versus informal presentation of
actions

• Scientific presentation
• Diagrams will convey the mechanisms by which
  actions will result in energy savings and a
  reduction in carbon emissions.
• Equations will convey the means by which energy
  savings and reductions in carbon emissions are
  computed for typical users.
• No mention will be made of individual preferences
  for some classes of actions over others, or the
  means by which individual lifestyle
  characteristics influence the impacts of various
  actions.

36
Scientific versus informal presentation of
actions, continued

• Informal presentation
• Scripted questions to subjects will determine
• Categories of actions are most important to them
• Constraints that limit consideration of certain
  action
• Motivation for pursuing energy reducing actions
• No mention will be made of amounts of energy
  saved for various actions, or the means by which
  impacts are computed.

37
Proposed data analysis

• Descriptive statistics
• Demographics
• Measures of decision-making styles
• Decision-making competency
• Decision-making outcomes
• Hypothesis tests
• Impact on decision-making outcomes of
• Intermediary type - intermediation type pairs
• Decision-making styles
• Decision-making competency

38
See you next year!
Questions?
39
References

• Bin, S. and H. Dowlatabadi. 2005. Consumer
  Lifestyle Approach to US Energy Use and the
  Related CO2 Emissions. Energy Policy 33 197
  208.
• Bruine de Bruin, W., Parker, A.M. and B.
  Fischhoff. 2007. Individual Differences in Adult
  Decision-Making Competence. Journal of
  Personality and Social Psychology 92(5) 938
  956.
• Häubl, G. and V. Trifts. 2000. Consumer Decision
  Making in Online Shopping Environments The
  Effects of Interactive Decision Aids. Marketing
  Science 19(1) 4 21.
• Lurie, N.H. and C.H. Mason. 2007. Visual
  Representation Implications for Decision Making.
  Journal of Marketing 71 160 177.
• Mankoff, J., Fussell, S.R., Johnson, M.P.,
  Matthews, D., Blais, D., Dillahunt, T., Glaves,
  R., McGuire, R., Setlock, L., Schick, A.
  Thompson, R. and H.-C. Wang. 2008. StepGreen
  Engaging Individuals in Energy-Saving Actions
  Online. Under review for presentation at
  Computer/Human Interaction Conference 2009,
  Boston, MA.
• Mankoff, J., Matthews, D., Fussell, S.R. and M.
  Johnson. 2007. Leveraging Social Networks to
  Motivate Individuals to Reduce Their Ecological
  Footprints, in Proceedings of the 40th Annual
  Hawaii International Conference on System
  Sciences (CD-ROM), January 3 6, 2007, Computer
  Society Press, 2007 (10 pages)
• Scott, S.G. and R.A. Bruce. 1995. Decision Making
  Style The Development and Assessment of a New
  Measure. Educational and Psychological
  Measurement 55 818 31.
• U.S. Department of Energy. 2006. Annual Energy
  Review 2005. Washington, D.C. Energy Information
  Administration, DOE/EIA-0384. World Wide Web
  http//tonto.eia.doe.gov/FTPROOT/multifuel/038405.
  pdf.
• van der Heijden, H. 2006. Mobile Decision Support
  for In-Store Purchase Decisions. Decision Support
  Systems 42 656 663.

40
Previous experiment choice sets
Heating/Cooling
41
Correlations User characteristics and decision
outcomes
42
Correlations Action representations and decision
outcomes