ACAL

1
ACAL Active Capture Automation Language

• Ana Ramírez
• Advisors Marc Davis, Jen Mankoff
• GUIR
• 25 February 2004

UC Berkeley - Garage Cinema Research - Group for
User Interface Research
2
Overview

• What is Active Capture
• Challenges in
• design
• Implementation
• Support at toolkit level for
• design
• implementation

3
Motivation

• Systems that direct human actions
• Keep awake system
• Sports instruction (golf swing)
• Automated health screening
• Video Door locks
• Interactions that control timing
• of interaction

4
Active Capture
Direction/ Cinematography
Capture
Interaction
Active
Capture
Human- Computer Interaction
Computer Vision/ Audition
Processing
5
Active Capture
Direction/ Cinematography
Capture
Interaction
Active
Capture
Human- Computer Interaction
Computer Vision/ Audition
Processing
6
Active Capture
Direction/ Cinematography
Capture
Interaction
Active
Capture
Human- Computer Interaction
Computer Vision/ Audition
Processing
7
Active Capture
Direction/ Cinematography
Capture
Interaction
Active
Capture
Human- Computer Interaction
Computer Vision/ Audition
Processing
8
Active Capture
Direction/ Cinematography
Capture
Interaction
Active
Capture
Human- Computer Interaction
Computer Vision/ Audition
Processing
9
Implemented Applications

• See Video at
• www.cs.berkeley.edu/anar/presentations/Implemente
  dApps.mpg

10
Head Turn Recognizer

• Uses
• Gross motion detector
• Eye detector
• Looks for
• No eyes and No motion followed by
• Motion followed by
• Eyes followed by
• No motion
• Uses mediation if something goes wrong.

11
Exercise

• Given raw materials
• User actions
• System actions
• Recognizers system uses
• Write down a representation of the interaction of
  the head turn recognizer.
• Work in groups of two

12
Designing Active Capture Applications

• Describe path of righteousness
• Describe what to do if something goes wrong
  (mediation)
• Use good mediation techniques
• Progressive assistance
• Freshness
• Graceful failure

13
Head Turn Application

• Path of righteousness is not obvious
• Tedious to write
• Expression of time flow cumbersome

14
Challenges

• Difficult to represent
• Control process with feedback (mediation)
• Timing
• Strict and non strict ordering

15
Goals of ACAL

• In general
• Natural to describe Active Capture applications.
• Support mediation strategies
• Include time flow primitives
• Support brain storming
• Visual language
• Support rapid prototyping
• Support for Wizard-of-Oz prototypes
• Support implementation
• Make path of righteousness apparent in code
• Be able to prove an implementation will reach the
  done state.

16
Current Status of ACAL

• Visual Language
• Started with Ka-Ping Yee in Marc Davis class on
  Multimedia Information in Spring 2003
• Toolkit level support
• Main focus this semester
• Support Wizard-of-Oz protoyping
• Future work
• Link visual language, toolkit and wizard-of-oz
  support together.
• Future Work

17
Current Research Areas

• Design Guidelines
• Jeff Heer, Nathan Good, Ana Ramirez, Marc Davis,
  Jen Mankoff. Presiding Over Accidents System
  Mediation of Human Action CHI04
• Language Support
• ACAL
• New Application
• Say Cheese

18
Visual Language

• Path of righteousness
• Observations
• Commands
• Capture
• Time Constraints

19
Visual Language

• Path of righteousness
• Observations
• Commands
• Capture
• Time Constraints

20
Visual Language

• Path of righteousness
• Observations
• Commands
• Capture
• Time Constraints

21
Visual Language

• Path of righteousness
• Observations
• Commands
• Capture
• Time Constraints

22
Visual Language

• Path of righteousness
• Observations
• Commands
• Capture
• Time Constraints

23
Visual Language

• Path of righteousness
• Observations
• Commands
• Capture
• Time Constraints

24
Visual Language

• Add mediation for case when actor is looking at
  camera before turn.

25
Visual Language

• Add freshness to mediation

26

• Add progressive assistance

27
Language Design Process

• Two key challenges
• Control-oriented vs. time-oriented representation
• Absolute vs. relative time relationships

28
Control vs. Time

• State machines and procedural programs describe
  control flow well
• ...but they visualize time poorly
• Timeline representation allows concurrency to be
  fully expressed
• ...but decisions and control flow dont fit
  easily on a timeline

29
Control vs. Time

• Hybrid visual representation timelines with flow
  arrows

30
Absolute vs. Relative Time

• Horizontal scale on timeline implies particular
  lengths of intervals
• Problem sometimes want ordering sometimes want
  specific intervals
• Solution arrangement on timeline yields
  ordering min/max specifiers constrain time
  intervals

31
Flexibility in Ordering

• Each point on a track specifies true, false,
  or dont care
• Example doesnt matter whether when waving or
  speaking begins or ends, as long as both happen
  at some point within a 5-second period

waving
speaking
lt 5 sec
32
Lessons Learned

• Difficult to balance between
• Control-oriented vs. time-oriented representation
• Absolute vs. relative time relationships
• Difficult to manage complexity
• Important to be able to see path of
  righteousness
• Easy to get mediation wrong.

33
Future Work

• Visual Language
• Better solution to absolute vs. relative time
  relationships challenge
• Better support for mediation strategies
• Implementation support
• Say Cheese
• Automated health screening

34
Questions

• anar_at_cs.berkeley.edu

35
System Architecture

36
ACAL Design Goals

• Natural to describe Active Capture applications
• Support key strategies for mediation
• Progressive Assistance
• Graceful Failure
• Freshness
• Include time flow primitives
• Support brainstorming process

37
ACAL

• Toolkit level support for applications with
• Mediation
• Complex timing
• Rich media input and output

38
Future Applications

• Say Cheese
• Folk Computing
• Support remote, more frequent medical screening.

39
Methodology

• Theoretical
• Active Capture Design Space
• Mediation strategies / guidelines
• Practical
• Reverse engineer implemented applications
• Design a new application