Human-Centered Computing Telepresence, AR, tangible interfaces

1
Human-Centered ComputingTelepresence, AR,
tangible interfaces

• John Canny
• UC Berkeley

2
Telepresence, AR, tangible interfacesJohn Canny

• PRoPs (Personal Roving Presences) and gesturing
  avatars.
• UPM - a universal planar manipulator.
• Bearable computers - turning laptops into
  augmented reality systems and wearable computers.

3
PRoPs Personal Roving Presences
Space Browser Eric Paulos
Surface cruiser
4
PRoPs Toward being there

• A PRoP is a Personal Roving Presence.
• PRoPs seek to bridge the gap between telepresence
  and being there.
• There are two essential aspects of remote
  presence that only telerobotics can provide
• Non-verbal communication, i.e. gaze, proxemics,
  gesture etc. which are highly spatialized.
• A gamut of human behaviors, such as chance
  encounters, informal meetings, showing or being
  shown, browsing, exploring etc, which are
  affordances of a physical presence.

5
Terrestrial PRoPs Carts

• Carts are a good match to human capabilities.
• Our strategy was to list important physical
  communication skills, and to implement as many as
  possible.

6
The rest of the storyA taxonomy of interaction
behaviors

• Verbal symbolic and prosodic
• Nonverbal communication theory (psychology,
  linguistics, anthropology). Main cues are
• Gaze
• Proxemics
• Facial expression
• Body and hand gesture
• Posture
• Touch
• Example A Handshake depends on all the cues.

7
Differences between verbal and nonverbal
communication

• Adam Kendon (83) summarizes the difference very
  well ...gesture has properties different from
  speech. In particular, it employs space as well
  as time in the creation of expressive forms,
  whereas speech can use only time. We find,
  therefore, that the way information may be
  preserved in speech, as compared to gesture,
  tends to be very different.

8
Why nonverbal skills are spatial

• Gaze - very sensitive to mutual direction.
  Center-right-left.
• Proxemics - Depends on distance to others and the
  surroundings.
• Facial expression - the least spatial, directed
  by gaze.
• Gesture - deictic gesture is completely spatial,
  so are other pictographic forms.
• Posture - Directional.
• Touch - Cant fake it without space.

9
You need social context as well

• Communication.
• Persuasion.
• Trust-building.
• These are radically different contexts that rely
  on different non-verbal cues. We need to study
  telepresence in representative scenarios.

10
The two-way street

• PRoPs provide selective control over particular
  cues. They can be turned on and off at will.
• So they provide a potentially interesting testbed
  for fine-grained study of non-verbal cues.
• We can create a rich variety of media and study
  their influence on the overall perception of the
  other person.

11
The input side

• How do you achieve rich, expressive input for
  PRoPs/avatars with todays technology?
• Cross-modal mapping from pen gesture to body
  gesture. Both have symbolic and expressive
  character

12
Future research?

• Gaze cue repair Face to face is not perfect.
  There are common breakdowns in face-to-face
  communication, such as gaze aversion.
• Beyond Being There It may be possible to rectify
  gaze aversion using image processing and/or a
  synthetic gaze cue.

13
Project 2UPM A universal planar manipulator

• A smart desk that can move a large number of
  objects placed on it independently.
• Provides a tangible interface in both directions
  for 2D layouts floor plans, landscapes, stage
  directions etc.
• Uses a small numbers of motors (4) and the
  non-linearity of friction.

14
UPM A universal planar manipulator

• First prototype used linear voice-coils from
  ancient disk drives, capable of about 2g
  acceleration. New prototype should manage about
  50gs.

15
Bearable Computing (new project)

• An exploration of issues in personal, persistent
  computing (augmented reality, worn interfaces)
  using ordinary laptop computers.
• Avoid head-mounted displays (expensive and
  low-res) head-tracking, and cables.
• The approach use optics to overlay computer
  images on reality, but use laptop or
  pocket-mounted displays.
• Testbed Grad course in HCC next semester.

16
An augmented reality classroom
Virtual image with notes, questions, private chat
space
Glasses to rotatethe laptop image
Physical space
17
An AR classroom

• Students work in groups of 5-7, communicating
  silently via pen or keyboard chat.
• Each group has one main note-taker, the others
  add their own comments or questions to the
  transcript.
• Students can mark up the group transcript, or the
  lecturers notes. There is non-archived chat
  also.
• One student per group works as facilitator or TA,
  posing questions to the others, and testing
  understanding.

18
Bearable computers on the go

• The laptop can serve as a bearable computer,
  attached wirelessly to its owner.

Double-mirror element in glasses
Heads-up virtual image
Pocket-worn portable LCD TV
Laptop in briefcase with wireless TV and
keyboard TXs.
Wireless chordal keyboard or Palmpilot in pocket
for input.
19
Goals of this research

• Can we enhance attention in class with live group
  chat/note-taking?
• Do we get a richer transcript with collaborative
  note-taking?
• Does the head-up display make note-taking less
  distracting?
• Will people do more tasks with a mobile bearable
  computer because the threshold for computer use
  is lower?