REALITY SHOW OR TRUSTABLE IMPOSTORS ?

1
REALITY SHOW OR TRUSTABLE IMPOSTORS ?

• Alex Garcia-Alonso
• University of the
• Basque Country

2
Outline

• Research environment
• Reality show or trustable impostor
• Recent research activities
• Your input discussion

3
Research environment

• Basque Country
• 4 Universities (90,000 students)
• 12 Tech research centers (2,200 labor force)

4
_at_ CEIT (1982-1995)

• Technological Research Center
• App. Mechanics Dpt. comp. graphics
• FEM pre- post-processing
• Interactive 3D visualization
• Mechanisms analysis
• Dynamics
• Collision analysis (RT mechanisms)
• Working for different companies
• European Space Agency
• MDI (Mechanical Dynamics Inc, Ann Arbor MI)
• EADS

5
_at_ Univ. of the Basque Country (1996- )

• Teaching !!!
• Technological Research centers
• CEIT (258 Labor staff)
• University of Navarre
• Ikerlan (230 Labor staff)
• MCC Cooperative group 250 companies -- 83,600
  labor force
• 44 Basque Country 37 Spain 19 international
• Labein (356 Labor staff)
• Vicomtech
• EUVE
• Spin-offs VR

www.landersimulation.com
www.simtechniques.com
6
Research environment
U. of the Basque Country U. of Navarre CEIT
Vicomtech
U. of Deusto Labein
Bilbao
San Sebastian
100 miles diameter
Mondragon
MCC Ikerlan U. of Mondragon
Vitoria
EUVE
7
SKILLS research network
Multimodal Interfaces for capturing and Transfer
of Skill

• CEIT
• Labein
• Aalborg University
• Commissariat à l'Energie Atomique - Laboratoire
  d'Intégration des Systèmes et des Technologies
  (CEA LIST)
• DLR - Institute of Robotics and Mechatronics
• Fraunhofer Institut für Graphische
  Datenverarbeitung
• HAPTION
• KUKA Roboter GmbH
• OMG plc
• PERCRO, - Italy coordinator
• Queen's University of Belfast
• Simonazzi S.p.A.
• Technion - Israel Institute of Technology
• UM1 - University Montpellier-1
• University of Tampere

8
ENACTIVE research network

• Labein
• CEIT
• Scuola Superiore Sant'anna, PERCRO
• Queen's University of Belfast Institut National
  Polytechnique de Grenoble
• University of Exeter
• Lunds Universitet
• Université Technologique de Compiègne
• Uppsala Universitet
• Institute of Robotik und Mechatronik
• Hasselt University
• Faculty of Music, McGill Univ.
• University of Minnesota

• Max-Planck Gesellschaft zur Foerderung der Wiss.
  E.V.
• Université de Genève
• Ecole Polytech. Fed. Lausanne
• Università Degli Studi Di Padova 
• Sony France
• Ecole des Hautes Etudes en Sciences Sociales
• Assoc pour la Création et la Recherche Outils
  d'Expression
• Centre National de la Recherche Scientifique
• Université Montpellier-1

Learn by doing
9
CEIT
10
Labein
11
Recent activities

• P2P visualization NURBS (2000-02, U. Basque
  Country)
• Collisions in mech. Assembly (2001-05, CEIT)
• Occlusion culling (2003-07, CEIT)
• Improving haptic perception (2006---, CEIT)
• Collaborative haptics (04-07, Labein)
• Avatars (04---, Vicomtech)
• Learning Procedural Skills link IA VR
  (07---, U. Basque Country - U. Navarre)

12
Let us go to the past 1986
13
Outline

• Research environment
• Reality show or trustable impostor
• Recent research activities
• Your input discussion

14
Reality show or trustable impostor

• Applications
• Users, costumers
• Reality show ?
• Impostors interface contract
• Trustable impostors everywhere
• Great Expectations
• Consumer haptics
• Networked haptics
• Summing up

15
Applications

• Research
• Physic based research
• Application oriented research
• Physics and Maths are the basis of research
• Science must also appreciate Applications
• Appraise applied science and engineering !
• Steer knowledge to a good application !

16
Applications spreadsheets

• David Alan Grier (1979)
• This machine has everything you can find on a
  real mainframe multiprocessing, report
  generators, Cobol. If it were only a small
  mainframe, there would be no reason to get it,
  because it is as slow as bad plumbing. However,
  it has one piece of software. This, my friend, is
  a spreadsheet and it is going to change the
  world. Without spreadsheets, these machines are
  nothing. With spreadsheets, they are
  unstoppable. (source, Computer, sept 2007, pg 6)

17
Applications users

• Windows interface metaphor
• Xerox
• Apple
• Users
• Apps target user domains of different size
• Specific domains (training, simulation, modeling,
  )
• Broad domains (Consumer oriented games,
  desktop I/F, co-presence, )
• Always
• pleased users are the key for successful
  applications

18
A personal experience discovering customers

• A sad story ADAMS/Real -Time Kinematics
• 1994 3 weeks from CEIT to MDI
• Jeff Cooper (MDI, Mechanical Dynamics Inc, Ann
  Arbor MI)
• The happy one ADAMS/Animation
• I learnt a new point of view customers
  (users)
• 2-3 K licenses

19
(discovering customers)

• Both applications (products) were
• Easy of use
• Quite good cost / performance ratio
• So, why this success difference?
• ADAMS/Animation
• conquered users expectations

20
REVIMA (developed at CEIT)

• 2000-04 Collaboration with CEIT
• Installed used successfully
• However, it did not replace the space mouse
  (haptic competitor) ? desk is more comfortable
• IEEE Computer Graphics Applications Dec. 2004
• Minority Report Steven Spielberg 2002 film

www.donnarosenartists.com artist Dale Rutter
www.ceit.es
21
Reality Show ?

• Is the realism of visual or haptic rendering the
  key for success ?
• Who generates the most realistic images ?
• Sonys Playstation 3 ?
• Microsofts Xbox 360 ?
• Nintendos Wii ? clearly, doesnt
• So, why did the Wii succeed ? ..
• Why do users enjoy more the Wii ?
• The Wii conquered users unconscious wishes
• How ?

22
Impostors lead the Wii to success

• Does the Wii replicates reality ?
• Games for the Wii Trustable Impostors
• An impostor replaces something
• Replaces a real golf club, baseball bat,
  bowling ball, racket
• Users trust (accept) their behavior

23
Impostors replacement restrictions

• There is not an absolute replacement
• Impostors can be used to achieve some goals
• They do not replace playing grounds
• These are not flaws, they are an Interface
  Contract
• Impostors are accepted as they are, because
  they fulfill users' expectations

Insertar FOTO juego real
24
Trustable Impostors everywhere

• Real time Computer Graphics
• Fluid flows
• Cloth
• Faked shadows and reflections
• Herd motion
• Lighting
• Vegetation
• Simulation
• Gravity used to fake acceleration

25
( Trustable Impostors everywhere )

• Haptic systems
• Deformable environments body tissue
• Comfortable inspection
• Pseudo-haptics
• Impostors everywhere
• Models substitute (real) entities
• Algorithms substitute physical laws
• Mathematical formulations
• Heuristics

26
Interface Contract for haptic systems

• Impostors are conditioned by hardware
  specifications, models and algorithms.
• Stiffness (mechanical apps)
• Force limits (weight, tighten, push, ...)
• Position precision
• Haptic systems require interface contract
• Conditions of use (usability domain)
• ?? analyses must be performed !!!
• Perception
• Skills achievement

27
90s Great Expectations 3D Visualization

• 1991 Jim Clark (Silicon Graphics)
• He also made some interesting predictions for
  the low end. He described the multimedia machine
  of the future as a device integrating the
  capabilities of (1) full 3D, interactive,
  real-time computer graphics, (2) full motion
  video, (3) image processing capabilities, (4)
  fully integrated geometric text, and (5) full
  compact-disc digital sound. He predicts that this
  will be available in a portable notebook-sized
  device with motion sensors in the next five to
  ten years for under 5,000. (Source Neville
  Holmes, Computer, sept 2007, pg 11)

28
Great expectations consumer haptics

• A haptic device as spread and affordable as the
  Wii or the GPU ?
• 3 dof seems available
• 6 dof and multipoint devices are further away
• Which performance ?
• Usability domain
• Which user expectations will they be able to
  fulfill ?
• ? Which applications should be developed ?

29
Great expectations networked haptics

• Feb. 2007 Sipping beers with Jarek Rossignac
• Q- collaborative haptics ?
• Thats OK.
• But which is its use ?
• A- Mechanical assembly
• (Jarek skeptical glance)
• A- feel mutual contact
• (Jarek keen glance)
• Co-presence awareness
• Toast, handshake,
• Mutual contact
• Feel physical presence in the collaborative
  working environment

30
Reality show AND trustable impostors

• We must discover unconscious users expectations
• And fulfill them
• Reality must be defined by an Interface
  contract
• Users must trust (accept) app. behavior
• ? Conditions of use must be well specified
• ? Perform more perception and skills
  achievement analyses
• Cheap haptic devices make possible new challenges
• May new haptic based metaphors replace the
  desktop ?
• Can haptic devices outperform the mouse ?
• Which I/F mechanisms can increase co-presence
  awareness ?

31
Outline

• Research environment
• Reality show or trustable impostor
• Recent research activities
• Your input discussion

32
Recent activities

• P2P visualization NURBS (2000-02, U. Basque C.)
• Intell. tutors 3D visualization (00-05, U.
  Basque C.)
• Collisions in mech. Assembly (00-04, CEIT)
• (1) Occlusion culling (03-07, CEIT)
• (2) Collaborative haptics (04---, Labein)
• Avatars (04---, Vicomtech)
• (3) Improving haptic perception (06---, CEIT)
• IA VR Learning Procedural Skills (06---,
• 
  U. Basque C. - U. Navarre)

33
(1) _at_ CEIT - Simulation group
Length 5.0 feet (1.5 m) Radius 2.6 feet (0.8
m) ITP supplies turbines to Rolls-Royce
34
Multidisciplinar group on haptics
2007 IEEE Int. Conf. on Robotics and Automation
2004 Comp Graphics Forum
2008 IEEE Trans. on Control Systems Technology
Haptic design building
2004 IEEE Comp Graph APP
2008 Computer Animation and Virtual Worlds
35
Occlusion culling environment

• Virtual environment
• Many objects
• Objects not far from eye
• Low occlusion
• Cannot use
• Conventional LOD
• Background texture replac.
• ? Improve occ. cull.

36
Occlusion culling learnt experiences

• Many coherence mechanisms analyzed
• Using most of them
• Surprisingly stereo (eye2eye)
• coherence did not improve
• performance why ?
• Low occ. scenes 40
• Speed ups achieved
• OC as a LOD mechanism ?

37
Occlusion culling learnt experiences

• OC as a LOD mechanism
• image relevance
• Occlusion ratio
• visible pixels / projected pixels
• A new GPU query is required
• GPU can be easily extended

Occlusion Ratio scheme if ( Occlusion_Ratio lt
threshold visible pixels lt
0.15 drawing area ) then draw simplified
object else draw object normally
38
(2) _at_ Labein

• Collaborative Assembly Simulator
• Labein (Spain) Queens University Belfast

HAVE2007 , Ottawa, Oct. 13-14
39
Learnt experiences

• Collaborative haptics (each user holds one
  object)
• Collaborative ? Cooperative two users hold the
  same object
• Independent interaction ? similar to
  visualization
• Dependent interaction
• Collisions ? co-presence
• Assembly constraints, network ? haptic response
• Experiments Labein - Queens U. Belfast
• P2P Scalability

40
Current research motion prediction

• Local haptic (not networked)
• 5 years ago 40ms/frame coll. computing
• Where will be the object when the collision
  computations are done ?
• Free movement vs. contact movement
• 
  instabilities ? ? stiffness dependent ?
• Currently 2-3ms. ? no relevant
• Collaborative haptics (50-200ms delays)
• promising

41
(3) _at_ CEIT

• Improving haptic perception
• Surface discontinuities (edges, holes, thin
  elements, )
• ? oscillations
• Surface normal
• Penetration
• Contact point

42
Your input discussion

• Thanks for your attention Muchas gracias
  Eskerrik asko (Basque language)

Donostia Saint Sebastian
43
Research background

• Networked visualization
• 1988-89 ? European Space Agency LAN TCP
• 1993-96 ? EADS ATM - UDP
• 1999-01 ? Basque Gov internet - P2P
• Haptic systems
• 2000-07 ? CEIT
• Networked haptic systems
• 2004-07 ? Labein, MIT, Queens U. Belfast

44
Labein
Dr. Rosa Iglesias, Ph D. Dec 2006 Labein
(2001-06) Ikerlan (2007-)