Title: When Users Pledge to Take Green Actions, Are They Solving a Decision Problem?
1When Users Pledge to Take Green Actions, Are They
Solving a Decision Problem?
Michael Johnson University of Massachusetts Boston Boston MA 02125 michael.johnson_at_umb.edu Susan Fussell (sfussell_at_andrew.cmu.edu) Jennifer Mankoff (jmankoff_at_cs.cmu.edu) Deanna Matthews (dh5x_at_andrew.cmu.edu) Leslie Setlock (lsetlock_at_andrew.cmu.edu) Carnegie Mellon University Pittsburgh PA 15213
INFORMS Fall Conference, Washington, D.C. October
15, 2008
2Acknowledgements
- Funding
- Intel Corporation, Leveraging Computational
Technologies to Support Behavior Change,
Jennifer Mankoff, principal investigator - National Science Foundation grants NSF
IIS-0745885 and NSF IIS-0205644, Jennifer
Mankoff, principal investigator - Research assistance
- Victoria Yew
3Problem Motivation
- Excessive energy consumption is a primary cause
of global warming - Americans consumed 100 quadrillion BTUs of energy
(U.S. Department of Energy 2006) - Energy consumption primarily linked to individual
activities lighting, heating and cooling (Bin
and Dowlatabadi 2005, U.S. Environmental
Protection Agency 2006) - Numerous Web-based initiatives exist to encourage
environmentally responsive behavior - Daily actions GreenSpeak.org
- Green travel EarthRoutes.net
- Carbon footprint calculations and green living
Yahoo!Green - Social networking sites enable individuals to
meet, collaborate and act collectively - Facebook
- MySpace
4Research questions
- Can we design an application to encourage
reductions in daily activities that affect global
warming that integrate - On-line social networking, to attract people who
are not necessarily green - Displaying progress towards energy reduction
goals, individually and in groups - Tracking actual energy consumption through
sensors - Multiple technology platforms
- Result is StepGreen, an initiative that combines
scholarship, outreach and social change - Mankoff, Matthews, Fussell and Johnson (2007)
- Mankoff et al. (2008)
5StepGreen A Web-based system in support
individual action to combat global warming
StepGreen.com actions for commitment
Social network site badge
Cumulative effects of actions taken
Source www.stepgreen.org
6Alternative disciplinary views of StepGreen
- Technology a flexible, robust system will induce
behavioral change and attract many users - IS/IT policy a popular on-line application will
provide novel insight into IT adoption, usage and
outcomes. - Decision sciences a Web-based decision support
system will help ordinary users make better and
more efficient decisions about high-impact daily
actions - Public policy learn about long-term social and
environmental impacts of individual change as
compared to national-level policy.
7Action selection is a decision problem
- Behavioral change is a multi-step process
- Persuade users that action on a particular topic
is urgent - Learn consequences of actions along multiple
dimensions - Explore alternatives by examining tradeoffs in
attribute space - Choose one or more actions that optimize utility
- Observe actual impacts of actions
- Update preferences for action attributes
8Action selection is a decision problem, contd
- Action representation is important to
decisionmaking - Present choices in lists of varying sizes
- One at a time
- List
- Present choices in differing ways
- Text descriptions
- Tables, charts and graphs
- Dynamics in action representations
- Static
- Interactive
What is the effect of length and information
content of alternative visual representations of
actions on decision time and decision quality?
9Theoretical Foundations
- Human-computer interactions (Dix, et al. 2004)
- Role of visual representation in decision-making
- Lurie and Mason (2007)
- Miller (2004)
- Mandel and Johnson (2002)
- Decision aids for large/complex decision problems
- Payne et al. (1988)
- Eiselt and Sandblom (2004)
- Decision support systems for consumer choice
- Häubl and Trifts (2000)
- van der Heijden (2006)
- Decision-making styles and barriers
- Bruine de Bruin, Parker and Fischoff (2007)
- Scott and Bruce (1995)
10Research gap decision aids and DSS for
public-sector problems
- Limited literature on decision support and
visualization especially by unsophisticated
users, or those in vulnerable or underrepresented
groups (Johnson 2006)
Can specific decision aids and visualization
strategies enable users to make decisions
regarding lifestyle choices more effectively, or
with higher levels of satisfaction?
11Previous experiment design
- Surveys
- Effects of human actions on global warming
- Decision-making styles
- Satisfaction with range of choices provided and
rankings made - Evaluate text-based actions according to
- Length (terse vs. verbose)
- Information content (relevant vs. irrelevant)
- Action category (e.g. Heating, Lighting,
Appliances, Water) - Actions are partitioned into two sets of
non-dominated alternatives - Superior (8 or 10)
- Inferior (2 of 10)
- Users rank top four actions out of 10 available
in four categories
12Previous experiment text representation
Water Consumption
13Previous experiment results
- Decision quality shows significant variation with
action categories - Few statistically significant associations with
decision length or quality - Question length
- Information content
- Decision style
- Satisfaction with the range of decision
alternatives - Satisfaction with the ranking decisions
- No evidence of learning about environmental
impacts
14Will alternative visualization methods make a
difference?
- Hypothesis users prefer graphical
representations of actions to text
representations and will make better decisions. - New design
- Four action categories
- Four visual representations of action
characteristics and impacts - Terse/relevant text
- Symbols
- Value path
- Bar chart
15Experiment data classifications, values
- Actions and categories
- Actions came from literature review and common
sense - Categories inspired by card-sorting exercises
- Appliances
- Heating/Cooling
- Lighting and Appliances
- Water Consumption
- Impacts
- Carbon emissions
- Change in energy usage Energy Star
(http//www.energystar.gov/) - Estimates of carbon savings Energy Information
Administration (http//www.eia.doe.gov/emeu/aer/tx
t/ptb1207b.html) - Dollar costs/savings Energy Information
Administration (http//www.eia.doe.gov/emeu/aer/tx
t/ptb0810.html) - Time costs/savings rules of thumb
- Quality of life subjective assessments
16Actions Text (Appliances)
17Actions Symbols
18Actions Value path
19Actions Bar chart
20Research framework, contd
- Users are assumed to act according to defined
decision-making styles (Scott and Bruce 1995) - Intuitive
- When I make a decision I trust my inner feelings
and reactions - Rational
- I make decisions in a logical and systematic
way - Dependent
- I often need the assistance of other people when
making important decisions - Avoidant
- I often procrastinate when it comes to making
important decisions - Spontaneous
- When making decisions I do what seems natural at
the moment - Users learn about environmental impacts of
various actions through choice process
21Hypotheses
- H 1 Participants decision quality and decision
time vary according to action category. - H 2a Graphical representation results in better
outcomes than text representations - H2b Decision time and decision quality varies
according to specific graphical representations - H 3a Rational decision-making styles are
associated with higher-quality decisions - H 3b Spontaneous decision-making styles are
associated with lower-quality decisions - H 3c Rational decision-making styles are
associated with slower decisions - H 3d Intuitive decision-making styles are
associated with more rapid decisions - H 4 Participants showed an increase in knowledge
about impacts of specific actions with respect to
global warming - H 5 Gender is associated with decision quality
and decision time
22Experiment design
- Within-subjects design four conditions (Martin
2004) - Survey software automatically randomized
presentation orders and recorded decision times - Action categories, graphical representations
counterbalanced across participants - 32 undergraduate and graduate student
participants - Steps
- Study overview and consent forms
- Pretest survey
- Action choices
- Posttest surveys
- Compensation
23Results Descriptive Statistics
- Participant characteristics
- 72 male
- 91 students (60 undergraduates 31 graduate
students) - 84.4 between 18 25 years old
- Most live in households with unrelated mates and
no children - Diverse racial, ethnic backgrounds
- Decision-making styles (means of 1 to 5
scaled question responses within categories) - Rational 3.82
- Intuitive 3.59
- Dependent 3.26
- Avoidant 2.68
- Spontaneous 2.87
24Results Descriptive Statistics, contd
- Baseline global warming knowledge is generally low
25Results Outcome measures
- Dominated responses number of choices that were
dominated in the subjects responses 0, 1, 2 - Response time time to make all selections used
log-transformed times due to skewed original
values
26Results Effect of action category
- Mixed model analyses showed no difference between
domains (appliances, heating, lighting, water)
for - Number of dominated choices (F 3, 36.3 1.71,
p .17) or - Log response time (F 3, 58.93 1.13, p .35)
No effect on decision quality or time due to
action category
27Result Effect of representation type
- Collapsed outcomes over action categories, giving
each participant one score for each visualization
condition - Dominated choices
- Log response time
Representation type has no effect on decision
quality Response times longer for text than for
graphics, and response times do not differ by
graphic type
28Results Pre- and post-test learning
After experiment, users generally perceived
greater global warming impact on all actions
29Correlations user characteristics and decision
outcomes
- Mean number of dominated choices per trial
negatively correlated with log of response time
( -0.405 0.021) - Gender (male 0, women 1) is negatively
correlated with log of response time (-0.439
0.012) - Decision-making style has no statistically
significant effect on decision-making quality or
decision time - Gender not statistically significantly correlated
with decision style or decision quality
30Correlations graphical representations and
outcomes
- Some relationships between decision times across
graphical representations - Text and value path (.470 0.008)
- Text and bar chart (.513 0.010)
- Symbols and value path (.603 0.000)
- Symbols and bar chart (.403 0.051)
31So, are StepGreen users solving a decision
problem?
- We think so..but decision context so far provides
little support. - For static representations of actions
- Generally, neither length of action descriptions,
information content within descriptions or
graphical representation of actions had
significant effects on decision outcomes. - Why did more information about decision
alternatives not help subjects make better
decisions? - Alternatives ranking too demanding cognitively?
- Information insufficiently tailored to different
needs, interests and backgrounds of subjects?
32Implications for research
- Design
- For maximum speed, emphasize graphical
representations of actions - Particular graphical representation not important
- Next steps
- Use a human intermediary to help users choose
actions - Apply new methods to measure decision-making
competence
33New Experiment Human-assisted decisionmaking
- Inspiration
- Risk-perception literature (Florig, et al. 2001)
detailed problem representation improves risk
ranking - Decision competence literature (Parker, Bruine de
Bruin and Fischoff 2007), in which measures of
decision efficacy are associated with
satisfaction with decisions made - Idea
- Use a human intermediary to help users better
understand their own values and preferences and
characteristics of action alternatives
34Goal evaluate decision outcomes along two
dimensions of interventions
- Intermediary types
- Peer intermediary - informal, youthful affect
and use a minimum of technical language - Expert intermediary - more formal, academic
affect, use technical language and appear to be
an authority on behavioral changes and impacts of
actions on climate change. - Intermediation type
- Quantitative - Problem-focused, scientific
presentation of the impacts of various actions
using figures and descriptions of relevant
calculations - Qualitative - individual-focused, interactive,
holistic discussion using probing questions to
learn about subject attitudes regarding different
actions
35Scientific versus informal presentation of
actions
- Scientific presentation
- Diagrams will convey the mechanisms by which
actions will result in energy savings and a
reduction in carbon emissions. - Equations will convey the means by which energy
savings and reductions in carbon emissions are
computed for typical users. - No mention will be made of individual preferences
for some classes of actions over others, or the
means by which individual lifestyle
characteristics influence the impacts of various
actions.
36Scientific versus informal presentation of
actions, continued
- Informal presentation
- Scripted questions to subjects will determine
- Categories of actions are most important to them
- Constraints that limit consideration of certain
action - Motivation for pursuing energy reducing actions
- No mention will be made of amounts of energy
saved for various actions, or the means by which
impacts are computed.
37Proposed data analysis
- Descriptive statistics
- Demographics
- Measures of decision-making styles
- Decision-making competency
- Decision-making outcomes
- Hypothesis tests
- Impact on decision-making outcomes of
- Intermediary type - intermediation type pairs
- Decision-making styles
- Decision-making competency
38See you next year!
Questions?
39References
- Bin, S. and H. Dowlatabadi. 2005. Consumer
Lifestyle Approach to US Energy Use and the
Related CO2 Emissions. Energy Policy 33 197
208. - Bruine de Bruin, W., Parker, A.M. and B.
Fischhoff. 2007. Individual Differences in Adult
Decision-Making Competence. Journal of
Personality and Social Psychology 92(5) 938
956. - Häubl, G. and V. Trifts. 2000. Consumer Decision
Making in Online Shopping Environments The
Effects of Interactive Decision Aids. Marketing
Science 19(1) 4 21. - Lurie, N.H. and C.H. Mason. 2007. Visual
Representation Implications for Decision Making.
Journal of Marketing 71 160 177. - Mankoff, J., Fussell, S.R., Johnson, M.P.,
Matthews, D., Blais, D., Dillahunt, T., Glaves,
R., McGuire, R., Setlock, L., Schick, A.
Thompson, R. and H.-C. Wang. 2008. StepGreen
Engaging Individuals in Energy-Saving Actions
Online. Under review for presentation at
Computer/Human Interaction Conference 2009,
Boston, MA. - Mankoff, J., Matthews, D., Fussell, S.R. and M.
Johnson. 2007. Leveraging Social Networks to
Motivate Individuals to Reduce Their Ecological
Footprints, in Proceedings of the 40th Annual
Hawaii International Conference on System
Sciences (CD-ROM), January 3 6, 2007, Computer
Society Press, 2007 (10 pages) - Scott, S.G. and R.A. Bruce. 1995. Decision Making
Style The Development and Assessment of a New
Measure. Educational and Psychological
Measurement 55 818 31. - U.S. Department of Energy. 2006. Annual Energy
Review 2005. Washington, D.C. Energy Information
Administration, DOE/EIA-0384. World Wide Web
http//tonto.eia.doe.gov/FTPROOT/multifuel/038405.
pdf. - van der Heijden, H. 2006. Mobile Decision Support
for In-Store Purchase Decisions. Decision Support
Systems 42 656 663.
40Previous experiment choice sets
Heating/Cooling
41Correlations User characteristics and decision
outcomes
42Correlations Action representations and decision
outcomes