Ink, Audio and Video - PowerPoint PPT Presentation

Loading...

PPT – Ink, Audio and Video PowerPoint presentation | free to view - id: 1415b8-MTRkO



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Ink, Audio and Video

Description:

Pen Computing. Use of pens has been around a long time ... Types of 'pens' Passive (same as using a finger) Active (pen provides some signal) ... – PowerPoint PPT presentation

Number of Views:46
Avg rating:3.0/5.0
Slides: 29
Provided by: georgep3
Category:
Tags: audio | ink | pens | video

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Ink, Audio and Video


1
Working with Natural Data Types
  • Ink, Audio (and Video)

2
Natural Data Types
  • As we move off the desktop, means of
    communication mimic natural human forms of
    communication
  • Writing, Speaking, Seeing
  • Ink, Audio, Video
  • Understanding these data types leads to
    interesting applications
  • e.g., capture applications

3
Uses of Natural Data Types
  • Recognition
  • Translate into other information (words or
    description of scene)
  • Dealing with errors
  • Raw data
  • as data use for human consumption (to read,
    hear or see)
  • Often linked with other context (time) in capture
    applications

4
Pen Computing
  • Use of pens has been around a long time
  • Light pen was used by Sutherland before Engelbart
    introduced the mouse
  • Resurgence in 90s
  • Types of pens
  • Passive (same as using a finger)
  • Active (pen provides some signal)

5
Example Pen Technologies
  • Passive
  • Touchscreen (e.g., PDA, some tablets)
  • Contact closure
  • Vision techniques
  • Active
  • Pen emits signal(s)
  • e.g. IR ultrasonic
  • Where is sensing? Surface or pen?

6
Questions about Pens
  • What operations detectable
  • Contact up/down?
  • Drawing/Writing?
  • Hover?
  • Pressure?
  • Modifiers? (like mouse buttons)
  • Which pen used?
  • Eraser?
  • Differences between pen and mouse?

7
Example Sony Ericsson Chatpen
  • Reads dot pattern on paper
  • - requires special paper
  • Transmits via Bluetooth

8
Example mimio
  • Active pens
  • IR ultrasonic
  • Portable sensor
  • Converts any surfaceto input surface
  • Can chain theseto create big surface
  • http//www.mimio.com

9
Pen input
  • Free-form ink
  • Soft keyboards
  • Recognition systems
  • generalize to gesture-based systems

10
Toolkits for Pen-Based Interfaces
  • UC Berkeley SATIN toolkit (UIST 2000)
  • http//guir.berkeley.edu/projects/satin
  • MS TabletPC
  • Descendant of MS Windows for Pen Computing
  • MS Pocket PC, CE.net
  • Apple Newton OS
  • GO PenPoint
  • Palm Developer environments
  • GDT (Long, Berkeley) Java-based trainable
    unistroke gesture recognizer
  • OOPS (Mankoff, GT) error correction based on
    subArctic

11
SATIN (UIST 2000)
  • Pen input for informal input
  • Sketching (others have investigated this)
  • Common toolkit story
  • Gee, X sure is a neat class of apps!
  • Golly, making X apps is tough!
  • Heres a toolkit to build X things easily!

12
The SATIN Toolkit
  • The application space
  • Informal ink apps
  • Beyond just recognition
  • Pen look-and-feel
  • Abstractions
  • Recognizers
  • Interpreters
  • multi-interpreters

13
Audio
  • Natural appeal of sound
  • Off-the desktop
  • hands-free
  • display not necessary
  • effective for a short distance

14
How sound is created
  • Sound is created when air is disturbed, usually
    by vibrating object(s) causing ripples of varying
    air pressure propagated by the collision of air
    molecules.

15
How sound is perceived
  • Characteristics of sound
  • Volume
  • Pitch
  • Features of sound wave
  • Amplitude
  • Volume amplitude of a sound wave
  • Frequency
  • Pitch frequency of a single sound wave

16
Example
17
When multiple sounds occur
18
Sampling sound
  • Note sampling rate affects accurate
    representation of the sound wave

19
Acquiring audio
20
Generating audio
21
Features in speech
22
Audio features
  • Pauses
  • Analogy to mouse up/down/drag
  • Who is speaking
  • Turn-taking
  • How is someone speaking
  • Prosody, affect
  • What is being said
  • Recognition of words

23
Challenges with Audio
  • Provide support for handling audio in manner
    similar to text
  • Simplify acquisition, segmenting, storing,
    playback
  • Open air audio (in/out) results in messy audio
    from ambient noise and feedback
  • Support noise reduction, enhance audio quality,
    echo cancellation

24
Noise reduction
  • Random noise is hard to predict

Noise Filter
f (t)
f(t)
25
Echo cancellation
  • Software hardware exist, but are hard for
    developers to easily add to application
  • Random noise is hard to predict, but echoes are
    not so random

Echo Canceller
f(t)
f (t)
26
Other interesting audio work
  • Auditory icons or earcons
  • sound that identifies objects actions
  • Automatic transformation of GUI interfaces to
    auditory interfaces
  • Mercator (Mynatt, 1995)
  • Sound localization sound cones

27
Creating UIs with audio
  • Audio
  • Java Media Framework
  • http//java.sun.com/products/java-media/jmf/
  • Speech
  • UC Berkeley SUEDE toolkit
  • http//guir.berkeley.edu/projects/suede/
  • CSLU toolkit
  • http//cslu.cse.ogi.edu/toolkit/

28
Next Time
  • Design Patterns for UI Construction
About PowerShow.com