Sound Software

1
Sound Software

• LT Mark Scorgie, USN

2
Topics

• Overview
• Difficulties
• Some solutions
• Questions

3
Overview

• Audio can increase realism
• Good audio increases perception of quality of
  video
• Audio is omnidirectional
• Audio important part of communication and
  environment perception

4
Difficulties

• Face similar difficulties as with visual
  rendering
• Sound modeling
• Sound synchronization
• Sound rendering
• Must be able to localize sound for realism

5
Some Solutions

• HRTFs
• Timbre Trees
• Beam Tracing
• Audio Ray Tracing

6
HRTF

• Head Related Transfer Functions (HRTF)
• Mathematical functions based on how you hear
• Measurements taken inside the ear canal
• Allows 3D sound for either headphones or
  speakers
• Modeling the listeners ears

7
HRTF
8
HRTF

• Ability to localize based on individual
• Can use non-individualized HRTFs
• Listen using someone else's ears
• Localization only if ears belong to a good
  localizer
• Can have errors
• Sounds directly behind/in front of listener

9
Timbre Trees

• Integrated system for modeling, synchronizing,
  and rendering sounds
• Works best with sounds that are parameterizable
• Difficult process for arbitrary sounds
• Primary consideration rendering sounds to keep
  up with frame rate

10
Timbre Trees

• Functional representation of sound
• Mapping of parameters
• Tree nodes contain
• Operators
• Representations of sounds
• Parameters
• Can be seen as an abstraction of a class of
  sounds

11

Examples of Timbre trees
12
Parameter Mappingexample
( loudness (combine ( (sine ( (rvector
200 20 10000) t)) (damp ( (rvector 200 20
10000) ( damping t))))))
13
Timbre Trees

• Rendering
• Trace sound within environment
• Process sound for effects (HRTF)
• Evaluate resulting trees

14
Rendering example
15
Beam Tracing

• Attempt to overcome computing reverberation
  paths
• Hinders real-time rendering

16
Beam Tracing

• S sound source
• Sa virtual sound source reflected around
  polygon a
• Ra reflected beam for which Sa is valid

17
Beam Tracing Method

• Four phases
• Spatial subdivision
• Beam tracing
• Path generation
• Auralization

18
Spatial Subdivision
d
k
d
k
a
a
e
e
b
b
j
l
j
l
f
f
c
g
c
g
m
n
m
n
q
q
i
i
p
p
h
h
Finished product cell adjacency graph
19
Beam Tracing
20
Beam Tree
21
Path Generation

• Use tree for interactive navigation through VE
• First, cell with R is found using log-time search
  of BSP
• Check each tree node for cell with R
• If yes, viable path. If not, disregard

22
Path Generation
23
Auralization

• Last step
• Once reverberation paths known, impulse response
  is generated
• Add one pulse for each distinct path
• Delay given by L/C
• Amplitude given by A/L

24
Audio Ray Tracing

• Approach to enhance 3D audio
• Processed offline through HRTFs
• Can be scaled for interactive environments
• Distribution of sound in air similar to light,
  but some major differences
• Sound has longer wavelength
• Can be received behind objects
• Speed is main difference for computation

25
Basic Approach

• Inputs
• Sound file
• 3D virtual room with 3D objects
• Set of HRTFs
• Location/direction of sound source
• Location/direction of listener

26
Ray Tracing Flow Chart
27
Audio Ray Tracing

• Recursive function
• Spawns rays until either
• Hits listener
• Max recursion depth reached
• For all rays that hit listener
• Load closest HRTF filter
• Combine with input sound file WRT length of path

28
Audio Ray Tracing

• Algorithm is scalable
• Important scalable parameters
• Number of emitted rays
• Number of HRTFs at receiver
• Length of HRTFs
• Sample frequency of sound
• Recursion depth

29
Audio Ray Tracing

• Most computation spent on processing HRTFs
• Unfortunately, an HRTF is associated with a
  single path
• However, allows for easy distribution over
  different machines for parallel processing

30
Interactive Audio RayTracing

• Define grid
• Compute pool of n 3D audio files for each
  intersection
• n 360/a, a increment of rotation angle
• When user moves through space, pool with closest
  grid position is selected
• From that pool, file with direction listener is
  moving to is played

31
Interactive Audio RayTracing
32
(No Transcript)
33
Bibliography

• Huopaniemi, J., Karjalainen, M.
• HRTF Filter Design Based on Auditory Criteria
• Hahn, J., Fouad, H., Gritz, L., Lee, J. W.
• Integrating Sounds and Motions in Virtual
  Environments
• Funkhouser, T., Carlbom, I., Elko, G., Pingali,
  G., Sondhi, M., West, J.
• A Beam Tracing Approach to Acoustic Modeling for
  Interactive Virtual Environments
• Mueller, W., Ullmann, F.
• A Scalable System for 3D Audio Ray Tracing

34
Question 1

• Name two of the three difficulties associated
  with adding sound to virtual environments.

• Sound modeling, synchronization, and
• rendering

35
Question 2

• What is the purpose for audio beam tracing?

• It is an attempt to overcome the difficulties
• associated with computing reverberation
• paths

36
Question 3

• Can audio ray tracing be used in an interactive
  enviroment?

• Yes