Tweek: Merging 2D and 3D Interaction in Immersive Environments

1
Tweek Merging 2D and 3D Interaction in Immersive
Environments

• Patrick Hartling
• Allen Bierbaum
• Carolina Cruz-Neira

2
Overview

• Problem description
• Goals for Tweek software
• Design overview
• Implementation overview
• Future work

3
Problem Description

• Interaction in VR is primarily 3D
• Not all interactions map well to this method
• Alphanumeric input
• Fine-grained interaction
• List selection
• Per-application custom interfaces
• Map abilities of specific devices to interaction
• Typically use 3D input devices
• Why abandon effective 2D interaction techniques
  in 3D space?

4
Tweek

• Framework for the development of cross-platform,
  cross-language, scalable GUIs
• GUIs communicate with remote applications
• GUIs can be use on the desktop, on a PDA, or in
  an immersive VR environment
• Part of the VR Juggler project
• Open source software for virtual reality
• Cross-VR system virtual platform for the
  development of VR applications
• http//www.vrjuggler.org/

5
Goals

• Allow GUI scalability
• Same GUI runs on the desktop, a PDA, and a
  projection-based VR system
• Must provide an easily extendable framework for
  cross-platform, cross-device GUIs
• Interact with VR applications written in any
  programming language
• Fully separate the GUI from the application
• Reuse existing GUI technology
• We do not want to re-invent scrollbars, menus,
  etc.
• Avoid cluttering immersive space

6
Design Overview (1 of 2)

• Observer design pattern
• VR application is the subject
• Maintains user-defined state information
• Per-application customizations
• GUI is the observer
• Provides a view of subjects state
• GUI interactions manipulate application state
• Benefits
• Distinct separation of application and GUI
• Allows multiple GUIs to observe the same
  application

7
Design Overview (2 of 2)

• Two-way communication between application and GUI
• Application can send data to the GUI
• GUI flexibility
• A GUI can load any component it needs to make
  interaction more effective
• Not limited to direct manipulation of application
  state

8
Implementation Overview

• Combination of technologies
• C for VR applications
• Java for GUI
• Basic Java framework
• Dynamically loaded JavaBeans for GUI components
• CORBA 2.3 for inter-language communication
• XML for JavaBean descriptions

9
Current Uses of Tweek (1 of 2)

• Interaction mechanism for military training
  application
• VCR-like controls for playback of pre-recorded
  session
• 6DOF navigation
• Entity data read-outs
• VR Juggler 2.0 configuration editor VrjConfig
• Makes extensive use of JavaBeans
• Remote reconfiguration of running VR Juggler
  applications

10
Current Uses of Tweek (2 of 2)

• Network data routing monitor PlxView
• Part of Plexus, the basis for VR Juggler 2.0
  collaboration software
• Provides multiple features
• View of the network
• Manipulation of network connections
• Charting of live network metrics
• Control of simulated network

11
Tweek in Use
12
Future Work

• Security
• Take advantage of CORBA security capabilities
  (SSL, TCP wrappers)
• Authentication is necessary
• Push GUI components to Java GUI from C
  application
• Applications give the GUI to the users
• Part of design but not fully implemented
• Interaction through web pages
• Make use of Java applets
• Incorporate a Python component

13
More Information

• Websites
• http//www.vrjuggler.org/tweek/
• http//www.sf.net/projects/vrjuggler/
• Mailing lists
• vrjuggler-info_at_lists.sourceforge.net
• vrjuggler-devel_at_lists.sourceforge.net