Windows audio architecture

1
Windows audio architecture

• Windows Driver Model
• supported by Win 98, Win ME, Win 2K and Win XP
• a single audio driver works for multiple Windows
  versions
• APIs
• DirectSound
• WinMM
• Kernel streaming
• multiple audio streams can be played at the same
  time
• SysAudio.SYS decides the optimal audio format and
  sample rate conversion
• Kmixer.SYS performs the actual converting

Win MM Application
DirectSound Application
WinMM.DLL
DSound.DLL
User Mode
Kernel Mode
SysAudio.SYS
Kmixer.SYS
Device Drive Container
Windows Components
By Hardware Vendor
USB Device Driver
IEEE 1394 Device Driver
PCI CARD Driver
ISA CARD Driver
2
WinMM API

• Simple, but
• high latency
• inability to take advantage of hardware
  acceleration
• no easy way to implement features, e.g. 3-D
  positioning, effect processing
• Play audio
• waveOutOpen() - open the output audio device
• waveOutWrite() - write the waveform audio data
• waveOutClose() - close the output audio device
• need to use callback or polling to check the
  result
• Not very interesting to real-time applications

3
DirectSound API - over view

• Audio component of DirectX package
• low latency
• use hardware acceleration
• direct access to sound device
• support capturing sound
• Two programming interfaces
• COM (Component Object Model) in C
• .NET in C, C, Visual Basic, etc.
• Important objects
• secondary buffers write/read audio data
• buffer cursors point to current captured/played
  audio data
• buffer notifications send events when buffer
  cursors reach a position

4
DirectSound API - COM interfaces

• IDirectSound8
• CreateSoundBuffer(descriptor, bufferPointer, )
• create a sound buffer object to manage audio
  samples
• fields of descriptor
• buffer size
• audio format commonly16 bits linear PCM
• buffer features
• SetCooperativeLevel(windowHandle, level)
• set the priority of the sound buffer

5
DirectSound API - COM interfaces

• IDirectSoundBuffer8
• Lock(offset, size, addr1, size1, addr2, size2,
  flag)
• ready all or part of the buffer for a data write
  and return pointers to which data can be written
• Play(reserved, priority, flags)
• cause the sound buffer to play, starting from the
  play cursor
• Unlock(addr1, size1, addr2, size2)
• release a locked sound buffer
• Stop()
• cause the sound buffer to stop playing

6
DirectSound API - COM interfaces

• IDirectSoundNotify8
• SetNotificationPositions(NumberOfNotifyStructure,
  ArrayofNotifyStructure)
• set the notification positions during playback,
  whenever the play cursor reaches one of the
  specified offsets, the associated event is
  signaled
• fields of NotifyStructure
• buffer offset
• notify event
• Sound capturing is similar

7
DirectSound API - code example

• Streaming audio in an event-driven thread

while (true) DWORD r WaitForSingleObject(eve
nt, INFINITE) // receives notification of
refilling buffer if (r WAIT_OBJECT_0)
Buffer.Lock(offset, size, addr1, size1,
addr2, size2, 0) // copy audio to buffer
addresses returned // by DirectSound //
could be two addresses because of buffer //
wrap-around memcpy(addr1, audio, size1)
if (size2 ! 0) memcpy(addr2, left,
size2) Buffer.Unlock(addr1, size1,
addr2, size2) // while
8
Windows audio architecture revisited

• Can we achieve lower latency?
• kernel mixing introduces at least 30 ms of delay
• kernel mixing is not necessary if Im the only
  application generating audio streams
• How about interacting with device drivers
  directly?

Win MM Application
DirectSound Application
WinMM.DLL
DSound.DLL
User Mode
Kernel Mode
SysAudio.SYS
Kmixer.SYS
Device Drive Container
Windows Components
By Hardware Vendor
USB Device Driver
IEEE 1394 Device Driver
PCI CARD Driver
ISA CARD Driver
9
DirectKS - the unofficial audio API

• Pros
• very low latency
• Cons
• only one application can play sound at one time
• applications need to handle audio format and
  sample rate conversion
• might not work in future version of Windows

Win MM Application
DirectSound Application
DirectKS Application
WinMM.DLL
DSound.DLL
DirectKS
User Mode
Kernel Mode
SysAudio.SYS
Kmixer.SYS
Device Drive Container
Windows Components
By Hardware Vendor
USB Device Driver
IEEE 1394 Device Driver
PCI CARD Driver
ISA CARD Driver
10
The next-generation Windows audio

• None of the current audio interfaces satisfies
  real-time applications
• transition between user mode and kernel mode for
  each I/O request
• blocking upon completion of an I/O request
• CPU cycles for copying data
• WaveRT (wave real-time) drivers in the next
  version of Windows - Longhorn
• data flow directly between the client and the
  audio hardware

11
Learn more

• URLs
• overview
• http//msdn.microsoft.com/library/default.asp?url
  /library/en-us/dnwmt/html/audiooverview.asp
• Windows Driver Model (WDM)
• http//www.microsoft.com/whdc/hwdev/tech/audio/wdm
  audio.mspxwdm1
• DirectKS
• http//www.microsoft.com/whdc/hwdev/tech/audio/Dir
  ectKS.mspx
• WaveRT
• http//www.microsoft.com/whdc/hwdev/tech/audio/Wav
  eRTport.mspx

12
Audio library overview

• Transmit audio over the internet
• use low latency audio APIs
• DirectSound or DirectKS
• pluggable codecs
• G.711, GSM, Speex, iLBC
• modular playout buffer
• integrated with rtplib
• System Requirements
• Windows XP or Windows 2K
• DirectSound 9.x runtime libraries
• Visual C runtime libraries

13
Audio library architecture
SIP user agent
SIP user agent
Audio tool GUI
Audio tool GUI
DirectSound/DirectKS
DirectSound/DirectKS
Playout buffer
Playout buffer
Encoder
Decoder
Encoder
Decoder
Rtplib
Rtplib
Socket
Socket
Network
Network
14
Audio library API

• Initialization
• setUserName(name)
• set the local user name
• setRemoteAddress(host/IP, port)
• send audio to this address
• setLocalAddress(host/IP, port)
• receive audio from this address
• setPlayerAudioFormat(audioFormat)
• play audio in this format
• setCapturerAudioFormat(audioFormat)
• capture audio in this format

15
Audio library API

• Initialization (Cont.)
• setEncoder(encoder)
• use this encoder to encode audio
• encoder can be created by
• encoder SpeexEncoder - create a Speex encoder
  instance
• encoder.setPayloadType(payLoadType) - set RTP
  payload type
• encoder.setOutputAudioFormat(audioFormat) - set
  the encoded format
• setDecoder(decoder)
• (similar to encoder)
• Start
• startReceiver()/startSender()
• start to receive/send audio

16
Audio library delay

• One-way mouth-to-ear delay measurement of audio
  library using DirectSound and DirectKS
• DirectKS shows close to 30 ms improvement over
  DirectSound