List of Multimodal Libraries

1
- List of Multimodal Libraries -
(EIF students only)
2
Toolkits review ( - EIA-FR)
R

• For EIF students
• A) Small demo ( Hello World  type)
• B) Provide a short documentation (3 copies) and a
  CD containing
• The library
• How to install the library
• What it does and how it works
• Quick state of the art of the specific domain, if
  applicable

3
TK1 EyesWeb !! Taken !!
R

• Target multimodal toolkit
•  EyesWeb consists of a number of integrated
  hardware and software modules which can be easily
  interconnected and extended. 
• Plate-forme Windows
• Suggested form review
• http//www.infomus.dist.unige.it/Index.html

4
TK3 HandVu !! Taken !!
R

• Target gesture recognition
•  Detects the hand in a standard posture, then
  tracks it and recognizes key postures - all in
  real-time and without the need for camera or user
  calibration. 
• Language C / Platform Linux, Win32
• Suggested form review
• http//www.movesinstitute.org/kolsch/HandVu/Hand
  Vu.html

5
TK5 Infovis Prefuse Toolkit
R

• Target visualisation toolkit
•  Interactive Graphics Toolkit written in Java to
  ease the development of Information Visualization
  applications and components. 
• Language Java / Platform all
• Suggested form review
• http//ivtk.sourceforge.net/
• http//prefuse.sourceforge.net/

6
TK7 iFace !! Taken !!
R

• Target 3D face animation, text-to-speech
•  Integrated framework for working with Face
  Multimedia Object by defining and implementing
  the necessary functionality  
• Language proprietary / Platform Win32
• http//ivizlab.sfu.ca/research/iface/

7
TK8 Sphinx 4.0 !! Taken !!
R

• Target speech recognition
•  state-of-the-art speech recognition system
  written entirely in Java 
• Language Java / Platform all
• Suggested form review
• http//cmusphinx.sourceforge.net/sphinx4

8
TK10 CLSU toolkit (2) !! Taken !!
R

• Target emotion recognition
•  A comprehensive suite of tools to enable
  exploration, learning, and research into speech
  and human-computer interaction. 
• Platform Win32
• Suggested form review
• http//cslu.cse.ogi.edu/toolkit/download/index.ht
  ml

9
TK11 Nouse Sonification Sandbox
!! Taken !!
R

• Target face tracking and sonification
• Nouse  Nose as a mouse  (face tracking)
• Simple demo programs/ Platform Win32
• http//synapse.vit.iit.nrc.ca/Nouse/index2.html
• Sonification Sandbox  simple, multi-platform,
  multi-purpose toolkit for sonifying data  
• Language java / Platform all
• http//sonify.psych.gatech.edu/research/sonificati
  on_sandbox/sandbox.html

10
TK12 CMU Communicator
R

• Target dialog manager
•  A dialog management tool based on Sphinx. 
• Language C / Platform Win32
• Suggested form review
• http//fife.speech.cs.cmu.edu/Communicator/

11
TK13 FestVox
R

• Target speech synthesis
•  The Festvox project aims to make the building
  of new synthetic voices more systemic and better
  documented, making it possible for anyone to
  build a new voice. 
• Language C / Platform nix
• Suggested form review
• http//festvox.org/

12
TK14 Other modalities review
R

• Suggested form review
• Review of possible other modalities
• including, but not limited to
• Eye gaze
• Face expressions
• Emotion recognition