Design and Implementation of a Wireless Handheld Multimodal Digital Video Library Client System

1
Design and Implementation of a Wireless Handheld
Multimodal Digital Video Library Client System

• Sam K. S. Sze
• Henry K. P. Choi
• Feb 27 2002
• Group Meeting

2
Abstract
We have developed a client system for accessing a
multimodal digital video library (DVL) namely,
iVIEW. It provides a user interface that meets
the challenge of rich multimodal information
presentation on wireless handheld device. An XML
schema is employed to organize the multimodal
metadata. Furthermore, we investigated a
context awareness mechanism complementary to the
XML schema to facilitate scalable degradation
under restricted resources in wireless
application environment.
3
Presentation Overview

• Introduction
• iVIEW Overview A Practical Perspective
• iVIEW Wireless Client
• Scalable Degradation
• Conclusion

4
Introduction

• Evolving DVL technologies
• Common challenges for wireless handheld device
  application development
• Challenges for implementing wireless digital
  video library client system

5
IntroductionEvolving DVL Technologies

• Multimodal information extraction and
  organization. The modals includes speech,
  on-screen characters, geographical locations,
  names, faces, on-screen objects
• Internet access
• Wireless access at anytime and anywhere

6
IntroductionCommon Challenges of Wireless
Handheld Devices

• Limitations Inherited from Hardware
• Screen Size
• Input Device (Pen device)
• CPU Power
• Memory
• Battery Life
• Bandwidth Limitations

7
IntroductionChallenges for Implementing a
Wireless Digital Video Library Client

• Intelligent User Interface
• Content Selection
• Result Set Refinement
• Integrate Handheld Platform Operation Manner
• Resources Management

8
iVIEW Overview
9
iVIEW OverviewIf Someone Comes to Room 121 and
Ask VIEW Technologies .
I want my video content
to be searchable over Internet and
wireless tomorrow. What kind of stuff
can you guys provide?
?
10
iVIEW OverviewWhats your answer? My Answer is

Demonstration
11
iVIEW Client Architecture
12
iVIEW Client iVIEW Client System Development

• The iVIEW Client has been developed in several
  approaches
• As a Microsoft Windows Native Application using
  Visual Basic as the frontend with a number of
  Visual C DLLs. (CMU version, Michael Christel)
  Further extended to support both English and
  Chinese. (April 2001, Sam)
• As a Java Applet for internet access using
  internal frames (Jacky FYP). Extended to support
  Real and Quicktime streaming, XML message parsing
  (August 2001, Sunny, Sam).
• As a web page for internet access using Dynamic
  HTML and ASP. (December 2001, Sam).
• Wireless

13
iVIEW Client A History of iVIEW Wireless Client

• April 2001 (1 month) Survey - Sam
• June 2001, (3 weeks) Wireless Web Version
  Gordon, Sam
• August 2001, (2 weeks) Framework of Multi-windows
  native application using Embedded Visual C -
  Sam
• October 2001 (2 weeks) Bluetooth and GSM
  compliance tested Sam, Mike
• November 2001 (3 weeks) A variant version with
  HTML viewer control Mike
• December 2001 (1 month) A multilingual search
  engine using ASP Sam, Gordon
• January 2002 (1 month) A full version of the
  multi-windows application - Henry
• February 2002 (2 weeks) A performance monitor
  added - Henry

14
iVIEW Wireless ClientImplementation

• Platform iPAQ PocketPC
• Wireless connections
• GSM HSCSD
• 2.5G CDMA
• 802.11b
• Bluetooth
• Multi-windows User Interface
• 3 Phrases of Operation
• Query
• Result Set Manipulation
• Presentation

15
iVIEW Wireless ClientQuery

• Chinese and English Query

16
iVIEW Wireless ClientResult Set Manipulations

• Segment View, Tree View, Topics View

17
iVIEW Wireless ClientPresentation
lt?xml version"1.0" encoding"big5" ?gt
ltsequence path"/iview/video/"gt lttime
start"0"gt   ltscriptgt GOVERNMENT HAS RESTORED
FULL lt/scriptgt   ltframe file"frame141_00.jpg"
/gt   lt/timegt lttime start"2"gt   ltscriptgt
DIPLOMATIC RELATIONS WITH LIBYA lt/scriptgt  
lt/timegt lttime start"4"gt   ltscriptgt AFTER A 15
YEAR SUSPENSION. CNN'S lt/scriptgt   ltframe
file"frame141_01.jpg" /gt   lt/timegt lttime
start"6"gt   ltscriptgt MARGARET LOWRIE HAS THE
DETAILS FROM LONDON   lt/scriptgt  
ltmapgtLONDONlt/mapgt   lt/timegt lttime start"8"gt  
ltscriptgt BRITISH FOREIGN SECRETARY ROBIN COOK
lt/scriptgt   lt/timegt lttime start"11"gt  
ltscriptgt ANNOUNCED DIPLOMATIC TIES WITH
LIBYA WOULD BE RESTORED BECAUSE IT
lt/scriptgt   lt/timegt lttime start"14"gt  
ltscriptgtIT NOW ACCEPTS RESPONSIBILITY
lt/scriptgt   ltframe file"frame141_02.jpg" /gt  
lt/timegt . . . . .   lt/sequencegt
18
iVIEW Wireless ClientGracefully Degradation at
Multimodal Presentation
19
iVIEW Wireless ClientGracefully Degradation at
Multimodal Presentation
20
iVIEW Wireless ClientGracefully Degradation at
Multimodal Presentation
21
iVIEW Wireless ClientEmbedded Resources Monitor

• Resources
• CPU Usage
• Memory Utilization
• Bandwidth Consumption

22
iVIEW Wireless ClientGracefully Degradation at
Multimodal Presentation
A set modal significance order chain (i.e.
S(m1) gt S(m2) gt S(m3)gt ) should also be
defined for different video types.
If system utilization saturates, the dispatch
will be signaled to make a decision to stop
dispatching a set of active modal of least
significance. Mathematically,
23
iVIEW Wireless ClientMeasurement A Typical
Client Operation Profile
24
iVIEW Wireless ClientMeasurement Degrade
Transition
25
Conclusion
A wireless DVL client system, that meets the
challenges of handling complex multimodal
presentations in a handheld device, had been
developed. We had shown a user interface design
that allow flexible content selection, result set
refinement in different views that fits the pen
computing operation profile, and it also equips
the capability of high complexity multimodal
presentation.   We define a multimodal
presentation XML with its corresponding context
aware scalable degrade mechanism that optimizes
the presentation content under resources limited
and fluctuating environment.
26
Q A