Audio design for interactive systems - PowerPoint PPT Presentation

Loading...

PPT – Audio design for interactive systems PowerPoint presentation | free to view - id: 549f6-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Audio design for interactive systems

Description:

the 500-pound gorilla if you need stuff like 8-channel output or linux/irix ... wide frequency band, complex attack. Loudspeaker array. Headphones with HRTF ... – PowerPoint PPT presentation

Number of Views:25
Avg rating:3.0/5.0
Slides: 26
Provided by: camillego
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Audio design for interactive systems


1
Audio design forinteractive systems
  • Camille Goudeseune
  • Integrated Systems Laboratory,
  • Beckman Institute

2
Outline
  • Sound libraries
  • music instruments
  • Digital Audio
  • music theory
  • Roles of sound
  • composition

3
high-level multiplatform (linuxwindows at least)
sound libraries
  • Java Sound API
  • VSS
  • FMOD
  • SDL
  • SEAL
  • Housemarque

4
Java Sound API
  • Painless for Java. Very basic. Avoid if you're
    ambitious. http//java.sun.com/products/java-media
    /sound/

5
VSS Virtual Sound Server
  • www.isl.uiuc.edu/software/software.html
  • the 500-pound gorilla if you need stuff like
    8-channel output or linux/irix/windows
    compatibility
  • steep learning curve for advanced stuff

6
FMOD
  • Syzygy (the Cube) uses it for sound.
  • CPU-miserly
  • finite but featureful API
  • www.fmod.org

7
SDL
  • large user community
  • open source.
  • Good if you want to integrate sound and graphics
    tightly.
  • www.libsdl.org

8
SEAL Synthetic Audio Library
  • many C compilers supported
  • Wide support for hardware acceleration
  • www.sonicspot.com/sealsdk/sealsdk.html

9
Housemarque
  • Multichannel with certain PC soundcards
  • www.s2.org/hmqaudio/

10
low-level sound libraries linux
  • OSS, www.linux.org.uk/OSS/
  • Most distros include the free basic version
  • 30 for fancy multichannel soundcard drivers
  • ALSA, www.alsa-project.org
  • religiously open-source alternative
  • 0.5.10 is the stable version
  • 0.9.0 development version is fine too(part of
    the upcoming 2.5 linux kernel).

11
low-level sound libraries windows
  • MMIO
  • simple, universal
  • waveOutGetDevCaps(), waveOutWrite(), ...
  • DirectSound
  • part of DirectX
  • LPDIRECTSOUNDBUFFER, CreateSoundBuffer(), Lock(),
    Unlock(), ...
  • faster but more awkward use a wrapper

12
Basics of Digital Audio
  • File formats
  • 16-bit or 8-bit?
  • These days, 8-bit is embarrassing.Pro gear uses
    32!
  • Stereo or mono?
  • Panning mono is faster simpler than stereo.
  • 8 kHz? 44 kHz?
  • .WAV, .AIFF, .AU, .MP3
  • converters sox and dozens of others

13
Basics of Digital Audio
  • Two tradeoffs

14
Basics of Digital Audio
  • Debugging
  • Gaps in sound ? excessive CPU use
  • stuttering ? CPU starvation(CPU fast enough
    but poorly scheduled)
  • Different from graphics! The frames per second
    cant degrade if the CPU is taxed.

15
Basics of Digital Audio
  • Debugging
  • Electric-guitar distortion ? clippingToo
    quiet hissToo loud clippingJust right
    almost clipping
  • For every stage in the audio pipeline,both
    software and hardware.Every place you can set
    the volume level!

16
Where to get sounds
  • Buy fx music libraries
  • Build record it yourself
  • Build synthesize it yourself
  • adjust an existing synth patch, a little or a lot
  • Steal websearch
  • 8-bit might still suffice while prototyping

17
What to do with the sounds,once you have them
  • common roles
  • alerts acknowledgements ambience
  • sound vs. image
  • speech vs. non-speech
  • synchronization with visual events
  • combining sounds
  • tips for spatializing

18
Short and subtle is best!
  • Graphical excellence gives to the viewer the
    greatest number of ideas in the shortest
    time with the least ink in the smallest
    space. (Edward Tufte, The Visual
    Display of Quantitative Information)
  • This applies to alerts, acks, and ambience.
  • Clutter is worse with sound no earlids!

19
Sound or image for a message?
  • Use sound if
  • Simple
  • Short
  • Standalone
  • Temporal
  • Demands immediate action
  • Use visuals if
  • Complex
  • Long
  • Referred to later
  • Spatial
  • Can deal with it later

20
Speech vs. non-speech
  • Main advantage precise, simple! But...
  • Carries extra connotations
  • Tone of voice, imprecise word choice, can confuse
    or distract
  • Non-speech doesnt interfere with conversation

21
Synchronizing to other things
  • Often 100 msec is accurate enough,when passively
    observing a sonic anda visual event
  • Sync with input action might needto be tighter,
    30 or even only 4 msec.

22
Moving a probe through a dataset
  • Trigger sounds on button-click, or when crossing
    thresholds
  • Or, play a continuous sound.
  • Vector-valued data is trickier.
  • Nominal data one sound per name.
  • Scope of probe point, ball, shell.
  • Auto-size based on probe speed.
  • Granulation

23
Navigating a VR world
  • Dense world
  • azimuth click rate varies with turning rate
  • altitude high/low beeps (rate
    varies with climb rate)
  • speed vehicle-engine metaphor
  • Large world quiet continuous ambiences
    localized to individual parts of the world.
  • Play only the nearest two or three.

24
Combining sounds
  • It can get only so loud before clipping.
  • Spatial separation (panning).
  • Temporal separation (one sound at a time).
  • A party of drunks / a good dinner conversation.
  • Frequency separation.
  • Each layer gets its own tempo.
  • Heavily layered techno or orchestral music.

25
Spatialized sound
  • Steady tones are worst.Bird chirps are best
    (they should know).
  • wide frequency band, complex attack
  • Loudspeaker array
  • Headphones with HRTF
  • Motion-tracked headphones
  • In the real world you move your head slightly to
    tell where a sound comes from.
About PowerShow.com