A Toolkit for Managing User Attention in Peripheral Displays

1
A Toolkit for Managing User Attention in
Peripheral Displays

• Tara Matthews, Anind K. Dey?, Jennifer
  Mankoff?, Scott Carter, Tye Rattenbury
• EECS Department, UC Berkeley
• Intel-Berkeley Research
• ?currently of HCII, Carnegie Mellon

2
Problematic Scenario

• Bus schedule
• Requires a task switch to use
• Doesnt afford continuous monitoring
• RTE LAWR GRIZ EUCL CNTRNUM HALL
  SR. GRIZ SHAT65 605a
  616a 630a 65 635a 646a 700a
  65 703a 716a 730a 65 735a
  746a 800a 65 803a
  816a 830a 65 835a 846a 900a
  65 903a 916a 930a 65 935a
  946a 1000a 65 1003a
  1016a 1030a 65 1035a 1046a
  1100a 65 1103a 1116a 1130a 65
  1135a 1146a 1200p 65
  1203p 1216p 1230p 65 1235p
  1246p 100p 65 103p 116p 130p
  65 135p 146p 200p 65
  203p 216p 230p 65 235p
  246p 300p65 303p 316p 330p
  65 335p 346p 400p

3
Solution Peripheral Displays

• Provide awareness with min attention
• Separate from primary task
• Bus LED Display
• Get bus info at a glance
• Dynamically manages attention it attracts
• Hard to accomplish

4
Why is creating PDs hard?

• Need to abstract info to be glance-able
• Need mechanisms for dynamically managing
  attention PDs attract
• Deciding attention levels to attract
  (notification levels)
• Displaying info appropriately (transitions)
• Our contribution
• Peripheral Display Toolkit (PTK) supports these
  key issues in PD creation

5
Example PTK Applications

• Remote Activity
• Social Guitar
• Audio Monitor
• Motion Monitor
• Remote Awareness Display
• Bus Displays
• Bus Mobile
• Bus LED
• Instant Messenger Status

Orb showing remote activity
Bus LED BusMobile
IM Picture Frame
Social Guitar
6
Example PTK Applications

• Remote Activity
• Social Guitar
• Audio Monitor
• Motion Monitor
• Remote Awareness Display
• Bus Displays
• Bus Mobile
• Bus LED
• Instant Messenger Status

Orb showing remote activity
Bus LED BusMobile
IM Picture Frame
Social Guitar
7
Example PTK Applications

• Remote Activity
• Social Guitar
• Audio Monitor
• Motion Monitor
• Remote Awareness Display
• Bus Displays
• Bus Mobile
• Bus LED
• Instant Messenger Status

Orb showing remote activity
Bus LED BusMobile
IM Picture Frame
Social Guitar
8
Example PTK Applications

• Remote Activity
• Social Guitar
• Audio Monitor
• Motion Monitor
• Remote Awareness Display
• Bus Displays
• Bus Mobile
• Bus LED
• Instant Messenger Status

Orb showing remote activity
Bus LED BusMobile
IM Picture Frame
Social Guitar
9
Example PTK Applications

• Manage attention with abstraction, notification
  levels, transitions
• Glance-able
• Few lines of code
• Simplified code design code re-use

Orb showing remote activity
Bus LED BusMobile
IM Picture Frame
Social Guitar
10
Simple Example Motion Monitor
no/low motion medium motion high motion
11
Outline

• PTK Motivation Examples
• Managing attention
• Abstraction
• Notification levels
• Transitions
• Toolkit Architecture Library
• Conclusion

12
Managing User Attention

• Attention is a crucial issue in design
• Significant impact on success
• More info can be monitored w/o overburdening
• Attention management is difficult
• Used to designing apps that attract full
  attention
• Attention attracted should match info urgency
• Literature surveys led to key characteristics of
  attention management

13
Attention Management Characteristics

• From survey of peripheral displays
• Abstraction
• enables lower-attention info consumption
• Notification levels
• help determine how much attention to attract
  based on urgency of info
• Transitions
• enable display of info to match urgency

14
Abstraction

• Enables lower-attention consumption of info
• Extracting features or reducing fidelity of info
• Motion Monitor replaces video images with
  general motion amount (high, medium, none)
• video ? abstraction ? amount of
• images motion

• high motion
• medium motion
• no/low motion

15
Notification Levels

• Indicate the urgency of info
• Low levels non-urgent info
• High levels more urgent info
• Motion Monitor doesnt show urgent info, so uses
  low notification levels

16
More on Notification Levels

• Attention is (roughly) scalar PD designers must
  decide where on the scale displayed info should
  fall

17
More on Notification Levels
Attention Type 1. None
18
More on Notification Levels
Attention Type 1. None
Notification Level Ignore
19
More on Notification Levels
Attention Type 1. None 2. Inattention
Notification Level Ignore
20
More on Notification Levels
Attention Type 1. None 2. Inattention
Notification Level Ignore Change blind
21
More on Notification Levels
Attention Type 1. None 2. Inattention 3.
Divided
Notification Level Ignore Change blind
22
More on Notification Levels
Attention Type 1. None 2. Inattention 3.
Divided
Notification Level Ignore Change blind Make aware
23
More on Notification Levels
Attention Type 1. None 2. Inattention 3.
Divided 4. Focused
Notification Level Ignore Change blind Make aware
24
More on Notification Levels
Notification Level Ignore Change blind Make
aware Interrupt
Attention Type 1. None 2. Inattention 3.
Divided 4. Focused
25
More on Notification Levels
Notification Level Ignore Change blind Make
aware Interrupt Demand action
Attention Type 1. None 2. Inattention 3.
Divided 4. Focused 5. Focused interaction
26
Transitions

• Enable display of info to match urgency
• Depend on notification level
• Ignore no changes
• Change blind fading, slow motion, tiny updates
• Make aware discrete, abrupt updates
• Interrupt flashing, beeping, vibration
• Demand action interrupt until user does action
• Motion Monitor varies pulse speeds for different
  notification levels

27
Outline

• PTK Motivation Examples
• Managing attention
• Abstraction
• Notification levels
• Transitions
• Toolkit Architecture Library
• Conclusion

28
PTK Architecture

• Support for managing impact on human attention
  using abstraction, notification levels, and
  transitions
• Simplified code design and code re-use
• Library of common PD components

Input
Abstractor
NotificationMap
Output
Transition
29
Motion Monitor Architecture
30
Motion Monitor Architecture
31
Motion Monitor Architecture
motion medium
32
Motion Monitor Architecture
motion medium
notification level change blind
33
Motion Monitor Architecture
motion medium
notification level change blind
34
Motion Monitor Architecture
5 lines of logic code
35
Remote Awareness Display
Count people,How much motion?
Phone ring?Voices?
36
Motion Monitor ?Remote Awareness Display
37
Motion Monitor ?Remote Awareness Display
38
Motion Monitor ?Remote Awareness Display
39
Motion Monitor ?Remote Awareness Display
40
Motion Monitor ?Remote Awareness Display
41
Motion Monitor ?Remote Awareness Display
42
Remote Awareness Display
54 total lines of code
43
General Architecture Diagram
44
Library Components

• Input
• audio, camera, Phidgets, Context Toolkit, online
  calendars, news, stocks, weather,Web page
  parser, serial port communication

45
Library Components

• Input
• audio, camera, Phidgets, Context Toolkit, online
  calendars, news, stocks, weather,Web page
  parser, serial port communication
• Output
• ticker text, Ambient Orb, Phidgets

46
Library Components

• Input
• audio, camera, Phidgets, Context Toolkit, online
  calendars, news, stocks, weather,Web page
  parser, serial port communication
• Output
• ticker text, Ambient Orb, Phidgets
• Abstractors
• motion, people counting, voices, phone ringing

47
Library Components

• Input
• audio, camera, Phidgets, Context Toolkit, online
  calendars, news, stocks, weather,Web page
  parser, serial port communication
• Output
• ticker text, Ambient Orb, Phidgets
• Abstractors
• motion, people counting, voices, phone ringing
• Notification
• exact match, threshold, contains,degree of
  change

48
Transitions

• PTK provides architectural support for
  implementing as animations
• Timing
• Number of steps
• Flashing (single, repeated)
• Tied to output
• Depend on output modality
• Example transitions color changing and pulsing
  (orb) fading, motion, flashing (ticker text)

49
Future Work Conclusion

• PTK supports the creation of PDs
• Focuses on managing user attention
• To do this, uses abstraction, notification
  levels, transitions
• Validated with 7 example applications
• Supports attention management
• Facilitates code re-use
• Future work
• Incorporate interruptibility andlocal context
  sensing
• Cross-modality animations
• PD evaluation support

50
Questions?

• For more information downloads
• www.cs.berkeley.edu/projects/io/ptk/
• tmatthew_at_cs.berkeley.edu
• Thanks to
• NSF, Ed De Guzman, Jeff Heer,Scott Klemmer,
  Berkeley CMU

51
What about user context?

• PDs must 2 sets of attention issues
• User context
• interruptibility, primary activity, focus of
  attention
• Attention management
• balancing info urgency and user attention
• PTK focuses on attention management
• User context invasive to sense, not practical to
  assume PDs have access
• Management issues are independent of user context
  
• Context input can be considered on a library
  level (in notification maps)

52
Related Work

• AROMA architectural support for abstraction
• Context Toolkit infrastructure for context input
  and some abstraction
• Phidgets library of hardware widgets for input
  and output
• Real World Interfaces Toolkit library of X10
  output widgets
• iStuff infrastructure for distributed inputs and
  outputs

53
Focus Attention

• Peripheral display attracting focused
  attention contradiction? Nope!
• Notifications enable displays to be peripheral
  when not displaying urgent info
• Attention is scalar PDs should cover entire range

54
PTK Validation

• Created 7 example applications that illustrate
• Supports attention management
• 3 used abstraction
• 5 used notification levels and transitions
• Easier to create
• 5-55 lines of logic code
• Facilitates code re-use
• used same input for 4 displays

Orb showing remote activity
Bus LED BusMobile
IM Picture Frame
Social Guitar
55
Notification Maps
Exact Match Notification Map
If event value equal to Set notification level
to no motion ignore med motion
change blind high motion make aware
(null) interrupt (null) demand action
56
Notification Maps

• Exact match
• checks event for equality w/specified values
• Threshold
• determines if event within specified ranges
• Contains
• checks if event contains specified strings
• Degree of change
• determines if event has changed by a specified
  amount since last event

57
Introduction

• Peripheral displays (PDs) enable you to monitor
  many info sources while maintaining a calm
  environment
• But, only calm if designed to manage the
  attention they attract
• PDs are hard to create
• No existing tools or guidelines for managing
  human attention
• Peripheral Display Toolkit (PTK)
• Enables easier creation of PDs that manage human
  attention

58
Why Peripheral Displays?

• Provide awareness w/ minimal attention
• Separate from users primary activity
• Non-focal display not used unless providing
  peripheral info
• Important to ubicomp vision of many devices to
  one user

59
Peripheral Display Toolkit (PTK)

• Supports the creation of PDs
• 1. Focuses on managing user attention
• 2. Simplifies code
• 3. Provides library of common components
• Managers user attention with
• Abstraction
• Notification levels
• Transitions

60
Characteristics of Attention Management

• Abstraction of info to be glance-able
• Dynamically managed attention
• Notification levels for deciding how much
  attention to attract
• Transitions for displaying info appropriately

61
Peripheral Display Toolkit (PTK)

• Supports the creation of PDs
• Focuses on managing user attention
• Balance info urgency and user attention
• To do this, uses
• Abstraction
• Notification levels
• Transitions

62
More on Notification Levels