A Toolkit for Managing User Attention in Peripheral Displays

1
A Toolkit for Managing User Attention in
Peripheral Displays

• Tara Matthews, Anind K. Dey?, Jennifer
  Mankoff?, Scott Carter, Tye Rattenbury
• EECS Department, UC Berkeley
• Intel-Berkeley Research
• ?currently of HCII, Carnegie Mellon

2
Introduction

• Peripheral displays (PDs) enable you to monitor
  many info sources while maintaining a calm
  environment
• But, only calm if designed to manage the
  attention they attract
• PDs are hard to create
• Managing human attention is ad hoc
• Peripheral Display Toolkit
• Enables easier creation of PDs that manage human
  attention

3
Example Audio Monitor
high volume medium volume no/very low volume
red / quicker pulse blue / medium pulse white /
slow pulse
4
Peripheral Display Toolkit (PTK)

• Supports the creation of PDs
• Focuses on managing user attention
• Balance info urgency and user attention
• To do this, uses
• Abstraction
• Notification levels
• Transitions

5
Why Peripheral Displays?

• Provide awareness w/ minimal attention
• Separate from users primary activity
• Non-focal display not used unless providing
  peripheral info
• Important to ubicomp vision of many devices to
  one user
• Example Motion Monitor

6
Why is creating PDs hard?

• Attention is a crucial issue in design
• Significant impact on success
• More info can be monitored w/o overburdening
• Attention management is difficult
• Used to designing apps that attract full
  attention
• Attention attracted should match info urgency

7
Outline

• Introduce the PTK
• Managing attention
• Abstraction
• Notification levels
• Transitions
• Toolkit Architecture
• Validation
• Conclusion

8
PTK Managing User Attention

• Abstraction
• enables lower-attention info consumption
• Notification levels
• help determine urgency of info
• Transitions
• enable display of info to match urgency

9
Abstraction

• Enables lower-attention consumption of info
• Extracting features or reducing fidelity of info
• Motion Monitor replaces video images with
  general amount of motion (high, low, or none)
• video ? abstraction ? amount of
• images motion

• high motion
• low motion
• no motion

10
Notification Levels

• Indicate the urgency of info
• Low levels non-urgent info
• High levels more urgent info
• Motion Monitor doesnt show urgent info, so uses
  low notification levels

11
More About Notification Levels

• 3 human attention types
• Inattention no conscious attention can affect
  behavior
• Divided attention distributing over several
  objects
• Focused attention all attention focused on one
  stimulus
• 5 corresponding notification levels
• Level Info urgency Attention
• Ignore unimportant ? none
• Change blind minimal ? inattention
• Make aware moderate ? divided
• Interrupt high ? focused
• Demand action critical ? focused perform
  some action

12
More About Notification Levels

• 3 human attention types
• Inattention no conscious attention can affect
  behavior
• Divided attention distributing over several
  objects
• Focused attention all attention focused on one
  stimulus
• 5 corresponding notification levels
• Level Info urgency Attention
• Ignore unimportant ? none
• Change blind minimal ? inattention
• Make aware moderate ? divided
• Interrupt high ? focused
• Demand action critical ? focused perform
  some action
• Peripheral display attracting focused
  attention contradiction?
• Nope! Notifications enable displays to be
  peripheral when not displaying urgent info

13
Transitions

• Enable display of info to match urgency
• Depend on notification level
• Demand action interrupt until user does some
  action in response
• Interrupt flashing, beeping, vibration
• Make aware discrete, abrupt updates
• Change blind fading, slow motion, tiny updates
• Ignore no changes
• Motion Monitor varies pulse speeds for different
  notification levels

14
Outline

• Introduce the PTK
• Managing attention
• Abstraction
• Notification levels
• Transitions
• Toolkit Architecture
• Validation
• Conclusion

15
PTK Architecture

• Support for managing impact on human attention
  using abstraction, notification levels, and
  transitions
• Simplified display design and code re-use
• Library of common PD components

16
Motion Monitor Architecture
5 lines of code
17
Motion Monitor Architecture
5 lines of code
18
Notification Maps
event value
Exact Match Notification Map
If event value equal to Set notification level
to no motion ignore low motion
change blind high motion make aware
(null) interrupt (null) demand action
19
Notification Maps

• Exact match
• checks event for equality w/specified values
• Threshold
• determines if event within specified ranges
• Contains
• checks if event contains specified strings
• Degree of change
• determines if event has changed by a specified
  amount since last event

20
Remote Awareness Display
Count people,How much motion?
Phone ring?Voices?
21
Motion Monitor ?Remote Awareness Display
22
Motion Monitor ?Remote Awareness Display
23
Motion Monitor ?Remote Awareness Display
24
Motion Monitor ?Remote Awareness Display
25
Motion Monitor ?Remote Awareness Display
26
Motion Monitor ?Remote Awareness Display
27
Remote Awareness Display
54 total lines of code10 put app together7
initialize GUI ticker8 orb wrapper29
ticker wrapper
28
General Architecture Diagram
29
Library Components

• Input
• audio, camera, Phidgets, Context Toolkit, online
  calendars, news, stocks, weather, Web page
  parser, serial port communication

30
Library Components

• Input
• audio, camera, Phidgets, Context Toolkit, online
  calendars, news, stocks, weather, Web page
  parser, serial port communication
• Output
• ticker text, Ambient Orb, Phidgets

31
Library Components

• Input
• audio, camera, Phidgets, Context Toolkit, online
  calendars, news, stocks, weather, Web page
  parser, serial port communication
• Output
• ticker text, Ambient Orb, Phidgets
• Abstractors
• motion, people counting, voices, phone ringing

32
Library Components

• Input
• audio, camera, Phidgets, Context Toolkit, online
  calendars, news, stocks, weather, Web page
  parser, serial port communication
• Output
• ticker text, Ambient Orb, Phidgets
• Abstractors
• motion, people counting, voices, phone ringing
• Notification
• exact match, threshold, contains, degree of change

33
Outline

• Introduce the PTK
• Managing attention
• Abstraction
• Notification levels
• Transitions
• Toolkit Architecture
• Validation
• Conclusion

34
PTK Validation

• Created 7 example applications that illustrate
• Supports attention management
• 3 used abstraction
• 5 used notification levels and transitions
• Easier to create
• 10-40 lines of logic code
• Facilitates code re-use
• used same input for 4 displays

Orb showing remote activity
Bus LED BusMobile
IM Picture Frame
Social Guitar
35
Conclusion

• PTK supports the creation of PDs
• Focuses on managing user attention
• To do this, uses
• Abstraction
• Notification levels
• Transitions
• Validated with 7 example applications
• Supports attention management
• Easier to create
• Facilitates code re-use

36
Questions?

• For more information
• www.cs.berkeley.edu/projects/io/ptk/
• tmatthew_at_cs.berkeley.edu

37
What about user context?

• PDs must 2 sets of attention issues
• User context
• interruptibility, primary activity, focus of
  attention
• Attention management
• balancing info urgency and user attention
• PTK scope limited to attention management
• User context invasive to sense, not practical to
  assume PDs have access
• Management issues are independent of user context
  
• Context input can be considered on a library
  level (in notification maps)

38
Related Work

• AROMA architectural support for abstraction
• Context Toolkit infrastructure for context input
  and some abstraction
• Phidgets library of hardware widgets for input
  and output
• Real World Interfaces Toolkit library of X10
  output widgets
• iStuff infrastructure for distributed inputs and
  outputs

39
Remote Awareness Display Code

• Abstractors as new TelephoneAbstractor(),
  new VoiceAbstractor()
• Abstractors cs new MotionAbstractor(),
  new PeopleCountAbstractor()
• InputSource ins new AudioInput(as),
  new CameraInput(cs)
• Output outs new ActivityTickerOutput(panel),
  new ActivityOrbOutput()
• String audio_ids RING, VOICE
• a_match new ExactMatchNotificationMap(
  audio_ids, false, null, true, null, null)
• c_match new ExactMatchNotificationMap(
  AMOUNT_OF_MOTION, MOTION_NONE, MOTION_LO,
  MOTION_HI, null, null)
• thresh new ThresholdNotificationSetter(
  NUM, new Range(0,0), new Range(1,999),
  null, null, null)
• NotificationMap map a_match,c_match,thresh
• makeConnection(ins, outs, map)

40
Example Applications

• Remote Activity
• Motion Monitor
• Remote Awareness Display
• Bus Displays
• Bus Mobile, Bus LED
• Ambient Audio
• Social Guitar, Audio Monitor
• Instant Messenger Status

Orb showing remote activity
Bus LED BusMobile
IM Picture Frame
Social Guitar
41
(No Transcript)