Title: Extensible 3D X3D Graphics Requirements for Video on the Web
1Extensible 3D (X3D) Graphics Requirements for
Video on the Web
- Don Brutzman
- W3C Workshop, Video on the Web
- 12 December 2007
- Web3D Consortium www.web3D.org
- Naval Postgraduate School
- Monterey California USA
2Topics
- X3D Overview
- Functional Requirements
- Video in X3D Scenes
- Converting 3D Models to Video
- Recommendations
3X3D Overview
4www.web3D.org
- The Web3D Consortium is a public-private
partnership of industry, agencies, universities
and individuals working on open standards for
real-time 3D communication on the Web. - Web3D develops, implements, evaluates and writes
the Extensible 3D (X3D) Graphics specification.
5Todays www.web3D.org page
6Community rules
- Thanks to steady innovation by Web3D members and
users, new features continue to evolve and grow
into X3D capabilities - Working groups identify requirements and write
implementable specifications - Individuals can directly extend specification
capabilities without formal permission
7ISO
- Formal annual review and ratification by the
International Standards Organization (ISO) have
made X3D an approved standard for real-world use,
both on and off the Web - Experts from 12-15 nations review our specs
- Immediate adoption by other governing bodies
helps to increase deployment
8W3C
- Further collaboration by Web3D Consortium with
the World Wide Web Consortium (W3C) has made X3D
a "first-class citizen" on the Web, providing
excellent (and growing) interoperability with
other XML standards - Current joint work includes efforts on Efficient
XML Interchange (EXI) for compression and
processing improvements.
9Intellectual Property Rights (IPR)
- Web3D and W3C have similar policies
- Any known patented technology must be declared by
members prior to consideration by working groups - Any patented technology contributions must be
licensed on a royalty-free (RF) basis for
inclusion in an openly used Web standard
http//www.web3d.org/membership - Caveat any legal problem can be solved, but
only in advance
10Open Source
- Open for any use, without license fees
- Free freedom to innovate
- Not necessarily free cost (more like free
puppy) - Common shared example implementation(s)
- Not a reference implementation the
specification/standard hopefully provides that
but required nevertheless - Open source implementations are necessary to
break logjams when company participants cant
resolve technical issues - Also provides self-sustaining business model for
continued activity, improvement
11IPR summary
- Royalties incompatible with Web architecture
-
- Open standards open source are both needed
- Open source by itself is necessary but
insufficient - Proprietary implementations are great too
- Standards organizations and IPR agreements
provide a stable playing field for long term
12X3D Profiles for Extensibility
- Different levels of content complexity
- Browsers can support increasing levels of
capability - Authors can use the proper palette for intended
delivery
MovieTexture node is Immersive Profile
13Example syntax
- ltShapegt
- ltBox/gt
- ltAppearancegt
- ltMovieTexture/gt
- ltTextureTransform/gt
- lt/Appearancegt
- lt/Shapegt
14Family of X3D specifications
- Abstract specification describes how it works
- Equivalent encodings
- XML .x3d, ClassicVRML .x3dv, Binary .x3db
- Scene Access Interface (SAI)
- Consistent programming in EcmaScript, Java
15X3D Specifications honeycomb diagram
16The key challenge is scalability
- Because the only information systems capable of
scalably growing to match global scope are the
Internet and the World Wide Web, X3D deliberately
follows the architectural principles of World
Wide Web. - Architecture of the World Wide Web, Volume One
http//www.w3.org/TR/webarch
17Functional Requirements
- Video within X3D Scenes
- 3D Geometry from Video
18Linkability
- Linkability
- Video content needs to be referencable by URI
- X3D approach allows ordered list of equivalent
URI addresses for reliable portability, access - Bookmark additions are desirable for video,
similar to using Viewpoints as X3D bookmarks - Further annotation of location or time as part of
URI is under consideration
19Planet 9 Raygun demos
raygun.planet9.com
20Playability
- Internal playability
- Video is often used as a user-facing billboard
within the X3D scene - Can be special display or match a particular view
- Mixed rendering is challenging but feasible
- Performance requirements are demanding
- External playability
- Anchor links or scripts may launch external video
displays for multimedia experience
21Interoperability
- Small number of required formats
- PNG JPEG GIF images, WAV audio, MPEG-2 video
- Motivated by performance and royalty-free (RF)
- Alternate formats are allowed but might not be
supported, url list can alleviate this problem - X3D browsers not (yet?) plugging in video codecs,
but plugin interoperability continues to improve - Adoption of imagery, audio and video formats
supported by W3C is shared goal for Web3D
22Planet 9 RayGun scene 1
Webcam stream as MovieTexture Billboard
23Animation
- Need synchronization of video playback with
3D-centric event-based behavior model - Necessary for consistent playability
- Connect simple X3D animation interpolators,
sequencers - Unlocks both Ecmascript and Java, either inside
X3D Script node or externally in HTML page - Future work assignment
- investigate possible alignment of X3D event model
with W3Cs Synchronized Multimedia Integration
Language (SMIL)
24Metadata overview
- X3D included document metadata (similar to html
meta tags) and typed Metadata nodes - Also Humanoid Animation (H-Anim) joint naming
- Some metadata conventions are emerging
- SAVAGE Modeling and Analysis Language (SMAL)
- No metadata vocabularies yet formalized by Web3D
- Wont reinvent other unencumbered standards
25Metadata requirements
- Video metadata of interest
- duration, loopability, whether file or streamed
- time, geographic location for precise positioning
in virtual environment - Camera direction (and perhaps settings)
- Ability to add other metadata extensions
- Metadata needs to be inspectable by browser
- Use cases available for video embedded within, or
external to, video content stream
26Construction of 3D geometry
- Current research demonstrating ability to
automatically generate 3D models from carefully
registered/recorded video streams - Variety of techniques are being developed
- Applications include city modeling, tracking
human pedestrians, classifying produced objects,
querying associated databases, etc.
27Planet 9 RayGun scene 2
Numerous efforts generate matching 3D geometry
using video sources
28Security
- Scene authors often want to encrypt or digitally
sign 3D content - Content is king, but creation is hard
- W3C recommendations for XML Security appear
appropriate encryption, signature, PKI - Security needs to be compatible with digital
rights management (DRM) scheme - Stream security needed, but not yet addressed
29Digital rights management
- X3Ds XML and Compressed Binary encodings allow
use of W3Cs Security recommendations - XML Encryption
- XML Digital Signature (for authentication)
- Public key infrastructure
- DRM appears to be feasible, but not used yet
- More uses exist than Hollywood-commercial
- See Suns DReaM project http//www.openmediacommon
s.org
30Streamability
- Numerous historic examples of streamed video in
3D environments - Streamed video in X3D browsers is feasible but
infrequent - Likely due to performance and format permissions
- Real Time Streaming Protocol (RTSP) might be
mapped to distribute X3D event model among
multiple players - Exemplar is Distributed Interactive Simulation
(DIS) protocol - Clear shared standards strategy might motivate
Web3D member support for necessary next-step
efforts
31Planet 9 RayGun scene 3
Browsers have ability to stream video within
scenes
32Synthesis
- Merging recorded or live video with 3D geometry
- Motion tracking for human avatars
- View from a window
- Likely future capability of broad interest
33Transparency
- Goal superposition of selected video pixels of
interest without box artifacts - Example video talking head or human avatar
without confounding background - Ability to encode some pixels as transparent
- Red Green Blue Alpha (RGBA) pixel encoding is not
practical for video streams - Designating certain color as transparent is
feasible, e.g. green screen
34Export
- Recording of animated activity in immersive
environment, as seen from one or more animated
viewpoints - Practice becoming more commonplace, (e.g.
video-game players save victorious sessions) - Different from playing back events within 3D
- Hollywood cliché What I really want is to
direct now becoming feasible - Similar requirements likely (e.g. video metadata)
35Recommendations
36Recommendations
- X3D graphics requirements for Web video are
technically and politically feasible - Interactive 3D performance is biggest challenge
- Plethora of oddball non-RF codecs unsustainable
- Royalty free model for video content thus
essential - Consider X3D technical requirements for video
when building the foundation for emerging
multimedia Web
37Contact
- Don Brutzman
- brutzman_at_nps.edu
- http//web.nps.navy.mil/brutzman
- Code USW/Br, Naval Postgraduate School
- Monterey California 93943-5000 USA
- 1.831.656.2149 voice
- 1.831.656.7599 fax