The MediaPhotonics project Photonics technologies for a virtual reality augmented reality system Ass

1
The MediaPhotonics projectPhotonics
technologiesfor a virtual reality augmented
reality system Assessment of Progress june 2005
2
Our aims

• Photonics
• Digitization (structured light, laser, from
  photos)
• Optical systems design, diffractive optics
• Light-Matter Interaction (Ray tracing, radiosity)
• Media
• 3D, Internet, mobile phone, computer graphics,
  video
• Mediaphotonics
• For a complete and generic Virtual
  Reality-Augmented Reality System using photonics
  technologies through the Internet or the mobile
  phone.
• Implement an E-learning tool based on this system

3
Applications

• Generic system
• Surgery
• Communication
• Light simulation (Pixium)
• Architecture Town Planning
• Distance learning (Optics-photonics, chemistry,
  surgery)
• A part of the courses will be available through
  the Internet (levelling, courses already taught
  for students to enroll in )
• Complex concepts presented in 3D
• Existing laboratory handling, machines
  demonstrations, etc. using 3D stereo video (film
  turning within labs or industry then video
  post-production)

4
Project partners

• Rhenaphotonics optics and photonics cluster
• European INTERREG III A program
• About 15 partners
• (Alsace, Bade-Württemberg, switzerland)
• Federating project MediaPhotonics
• LSP (Illkirch), FHO Offenbourg, Formatis (Basel),
  Kepilion (Kehl)
• Virtual Reality-Augmented Reality generic system
• 3D E-learning
• Ircad / Micromodule
• Augmented Reality helmet for surgery
• Pixium a spin-off of our activity creation
  april 2004
• Light simulation

5
Main functions and data
6
Digitization

• Digitization of objects
• Laser profilometry
• Structured light, phase shifting
• 3D digital model
• Real space reconstruction
• Using non-calibrated photos
• To insert virtual objects in this space
• Geometric registration of virtual space onto
  the real space for Augmented Reality

7
Lightning simulation
8
Tracking and interactions

• Head tracking
• Get the position and orientation of each eye
• 3D mouse
• Position and orientation
• Click and interactions with a virtual scene
• Technologies
• Leds cameras
• Electromagnetic field
• Our solution
• Inertial sensors (relative position and
  orientation)
• Gyroscope, accelerometer, inclinometer
• Ultrasound (absolute position and orientation)
• Flight time and phase detection, sensors fusion

9
Clients/server applications

• Through the Internet
• 3D data
• Server application
• Connections management
• Interactions management
• Client applications
• Interaction with the 3D data
• Vizualization with transmitted
• Virtual cameras parameters
• interactions
• Multi-users wherever in the world

10
Clients/server through mobile phone
Distant connection GPRS
Local connection Bluetooth
Visualization and interaction with 3D data
through mobile phone -gt toward the concept of
wearable computer
11
Display systems
12
Stereoscopic projection

• 2 projected images (with 2 shifted virtual
  cameras)
• Polarized light - 2 projectors (cross filters)
• Stereopsy rebuilt by stereo vision
• Glasses with polarizing filters
• Special projection screen

13
Virtual reality helmet

• Stereo visualisation
• LCD display
• Optical system with variable focal length
• Allowing the display of virtual pictures from 1
  to 3 meters from the user.
• Motorized optical lenses
• Optical design with Code V

14
Augmented reality surgery
virtual
Real
virtual
Real
Internal view
External view
Real and virtual mixed Collaborations
15
Augmented reality helmet
16
Stereo video