Studying The Use of Handhelds To Control Everyday Appliances

1
Studying The Use of Handhelds To Control Everyday
Appliances
Jeffrey Nichols Carnegie Mellon
University August 24, 2001
2
The Problem

• Interfaces to appliances are becoming more
  complex.
• Stereos, televisions, microwaves, alarm clocks,
  telephones, VCRs, etc.
• Little standardization among similar appliances.

3
The Solution

• Observations
• Many people now carry a handheld device.
• Mobile phone, Palm, PocketPC
• Handhelds have richer interface technology
• Large LCDs, touchscreens, text-entry
• Solution
• Move the interface from the appliance to the
  handheld!

4
The Personal Universal Controller

• The appliance and the handheld will have two-way
  wireless communication.
• Each appliance will have a built-in specification
  that is sent to the handheld, which then
  generates an interface.

Specifications
Control
5
Benefits

• The interfaces take into account
• user preferences
• conventions of the handheld device
• other interfaces that the user is familiar with

6
Outline

• A First Step
• User Studies
• Future Work

7
A First Step

• Build Reference Interfaces
• Remote control interfaces for various appliances
  that we design manually.
• Verify that better interfaces can be created on a
  handheld
• Used for understanding what functional knowledge
  is necessary to make a good interface.

8
Reference Interfaces

• Interfaces were hand-designed for two appliances
  and two handhelds
• Appliances
• AIWA CX-NMT70 Shelf Stereo
• ATT 1825 Telephone/Answering Machine
• Handhelds
• Palm
• Microsoft PocketPC

9
Palm Interfaces

• We initially designed paper-prototype interfaces
  for Palm

telephone
stereo
10
PocketPC Interfaces

• We later implemented interfaces for Microsofts
  PocketPC (simulated remote control).

telephone
stereo
11
Interface Quality?

• We iteratively improved the interfaces using
  heuristic analysis techniques.
• We conducted a think-aloud study with several
  Carnegie Mellon students to find problems in the
  interfaces.
• Lastly, we conducted a user study that compared
  our reference interfaces with the manufacturers
  interfaces.

12
Outline

• A First Step
• User Studies
• Future Work

13
User Studies

• Two Studies
• Study 1
• Paper-Prototype Palm vs. Actual Appliance
• Study 2
• Functional PocketPC vs. Actual Appliance

14
User Studies, cont.

• Procedure
• We did a between-subjects study.
• Each subject worked on two sets of tasks.
• In order to minimize subjects, each worked on
  both the stereo and the phone.
• We controlled for order and interface type.

15
Evaluation of Task Performance

• Three Metrics
• Time to complete all tasks
• Number of times help was requested
• How often did the subject need the manual or
  online help?
• Number of missteps
• Misstep the pressing of a button that does not
  advance the progress on the current task
• No missteps were counted after the user requested
  help.

16
User Study 1 PalmOS

• Paper-prototype study
• Good results
• Users made 1/5 as many errors and requested help
  1/2 as often with the handheld interfaces
• This was encouraging, but
• We wanted to verify these results with higher
  fidelity interfaces

17
User Study 2

• We implemented the interfaces on a handheld and
  simulated remote control of an actual appliance.
• Remote control applications built in Visual Basic
  on a PocketPC
• Control of stereo and phone simulated in software
• Feedback appeared to come from the actual
  appliance

18
User Study 2, cont.

• Participants
• Twelve students from Carnegie Mellon
• Four female, Eight male
• Volunteered in response to a newsgroup
  advertisement
• Paid 7 for their participation (30-45 minutes)
• All had limited handheld experience
• Half (6) owned Aiwa-brand stereos
• Two had ATT digital answering machines

19
User Study 2 Results

• All differences are significant (p

• About ½ the time and ½ the errors!

20
Qualitative Results

• Why were the reference interfaces better?
• Disabled elements were not always shown on the
  screen.
• Less-used functions could be hidden in menus or
  dialog boxes.
• Labels could dynamically change.
• Clear feedback and explanation of the current
  state was possible.

21
Outline

• A First Step
• User Studies
• Future Work

22
Future Work

• Build more reference interfaces
• We will create more interfaces to be sure that we
  understand as many of the features of remote
  controls as possible.
• Copy machine
• Microwave oven
• MP3 player

23
Future Work, cont.

• Design a specification language
• This is currently in progress.
• Information currently in the language
• State variables and commands
• Grouping tree
• Dependency graph
• Lots and lots of labels

24
Future Work, cont.

• Build an automatic interface generator
• Determine the structure of the interface
• Choose interface widgets for each state variable
  and command
• Layout the widgets on the screen

25
The End

• For more information see
• http//www.cs.cmu.edu/jeffreyn/
• http//www.cs.cmu.edu/pebbles/
• Or e-mail me at
• jeffreyn_at_cs.cmu.edu

26
(No Transcript)
27
Actual Appliance Interfaces

• Lots of Problems
• Poorly labeled and overloaded buttons
• Insufficient feedback
• Timer example
• Programming the speed-dial
• Phone has technical separation between phone and
  answering machine

28
Qualitative Results

• Grouping controls is important
• Groups define which elements are placed adjacent
  to each other and how elements are separated onto
  pages.
• Groupings vary between devices and interface
  styles.

29
Qualitative Results, cont.

• Dual-associated functions are hard to make
  obvious for users
• The record button is associated with both tapes
  (record onto) and each of the other modes
  (recorded from).
• Some users expected the first mapping to used,
  whereas the controller used the second mapping.

30
Qualitative Results, cont.

• Tree-based structures are not sufficient for
  fully describing an interface
• Some interface concepts, especially
  dual-associated functions, break the tree because
  they may interact with the children of several
  different elements within the tree
• The record button breaks the stereos tree
  structure because it is globally accessible but
  has different local effects.

31
Qualitative Results, cont.

• A single function may map to multiple interface
  widgets (and vice versa)
• Example One state variable could be used to
  represent all of the playback states of a tape
  player. The play, stop, fast-forward, and rewind
  buttons all act on this single variable.

32
Applying These Results

• We are actively applying these results to the
  design of the specification language
• A tree-grouping structure is augmented with a
  dependency graph to help describe dual-mapped
  functions
• Ranking relationships within groups using
  priorities
• We will also apply them in the design of the
  automatic layout engine

33
Future Work

• Build the specification language and automatic
  generation engine

34
A Hard Problem

• Automatically generating a good user interface is
  hard, but we think we can do it for several
  reasons
• Remote controls are a special class of user
  interface that use relatively simple interaction
  techniques.
• Buttons, text fields, and other standard widgets.
• Our approach differs from earlier work

35
The Approach

• Study Interfaces
• Functional knowledge of the appliance
• What must the appliance tell the handheld about
  itself so that a good interface can be
  constructed.
• Design and Layout
• How do we turn the knowledge about the appliance
  into a usable interface?
• Design a specification language
• Build an automatic interface generator

36
Our Progress

• Study Interfaces
• Functional knowledge of the appliance
• What must the appliance tell the handheld about
  itself so that a good interface can be
  constructed.
• Design and Layout
• How do we turn the knowledge about the appliance
  into a usable interface?
• Design a specification language (in progress)
• Build an automatic interface generator

37
User Study 1 Extra Slides

• Participants
• 13 Carnegie Mellon graduate students
• Five female, Eight male
• Enrolled in School of Computer Science
• Seven owned a Palm device
• One had no Palm experience
• Four owned Aiwa-brand stereo systems

38
User Study 1 Results

• Users made five times the errors and needed help
  twice as often with the actual appliances!
• All results significant (p for all)

39
Problems with User Study 1

• Paper-prototype study introduced a high
  possibility of experimenter interference.
• Solution
• Create an environment that completely simulates
  what one might experience using a personal
  universal controller
• Interfaces running on an actual handheld
• Interfaces should appear to control an actual
  appliance

40
James went to the bathroom

• He will return in a moment.
• 1116am