Voice Quality and Testing EIT-03 Alan Percy Director of Business Development AudioCodes

1
Voice Quality and TestingEIT-03Alan
PercyDirector of Business DevelopmentAudioCodes
2
Who is AudioCodes?

• Leading OEM Manufacturer of Enabling VoIP
  equipment
• Gateways
• Media Servers
• Boards, Chips and Modules
• Key IPR holder for G.723.1
• 11th Year in Business
• 82.8M Revenue for FY04
• NASDAQ AUDC
• Over 440 Employees
• Worldwide Company
• Headquarters in Israel
• US offices in San Jose, Boston, Chicago, Dallas,
  RTP, and Somerset NJ
• Offices in Mexico, France, UK, China, and Japan

3
Background

• Customer demand is building
• VoIP carriers are proliferating
• What will differentiate the carriers?

Source In-Stat
4
New Carriers, New Challenges

• A few years ago, we equipment providers thought
  we had this figured out.
• Then
• Private networks
• Intra-Enterprise
• Carefully controlled (over-engineered) networks

5
New Carriers, New Challenges

• Today New Frontier
• Depending on existing broadband
• Cable
• DSL
• Fixed wireless
• New Wireless Services
• 3G
• WiMax
• WiFi
• SAT

6
What have we learned?

• You cant always control the network
• To make the business model work, you sometimes
  need to leverage the existing infrastructure

7
What have we learned?

• The last mile is the hardest mile!

8
What have we learned?

• Murphys law is true!
• (Poor voice quality will only affect the CEO
  during an important investor call)
• Poor voice quality will get you kicked out
• Huge variation in voice quality between equipment
  vendors
• (Especially when network conditions are not
  ideal)

9
Is voice quality important?

• Voice Quality is the most sensitive issue for
  business customers Greg Schreiber, VP of
  Product Development - Vaspian

10
What affects voice quality?
Packet Loss
Echo
Jitter
Latency
Quality
Coder
Comfort Noise
Voice Activity Detection
11
Choosing the Right Coder
12
Vocoder Factors

• Single speaker, no packet loss, no acoustic noise
  
• Vocoder quality given non speech audio (music,
  background noise)
• Bit rate
• Algorithmic delay Packet size look ahead
• Audio bandwidth
• Low bit rate algorithm type
• Complexity ( MIPS)
• Intrinsic robustness to packet loss

13
Vocoder Factors

• Vocoder implementation issues
• Not all vocoders standards have bit exact test
  vectors
• And test vectors dont cover every angle
• Little known fact - even long approved and
  deployed vocoder standards still have bugs let
  alone new ones
• Audiocodes has found and reported bugs in
  numerous standard vocoders
• Hence the importance of Speech Quality Testing
  and deployment experience

14
Vocoder Tradeoffs

• Complexity is always a constraint
• Bit rate and algorithm type affect speech and
  non-speech vocoding quality
• Larger packet size enables lower bit rate but
  increases delay (e.g. G.723.1 vs. G.729A)
• Higher bit rate enables better intrinsic
  robustness to packet loss
• Higher audio bandwidth requires higher bit rate

15
Echo Canceller Factors

• Imperfect EC performance could result in
• Residual echo
• Clipping
• Distorted speech
• Annoying variation in background noise
• Longer round trip delay makes echo cancellation
  more critical

16
Echo Canceller Factors

• ECs are challenged most during Double Talk (DT)
• Double Talk happen occasionally during full
  duplex conversations
• During Double Talk
• EC may diverge
• The Non Linear Processor (NLP) could clip,
  attenuate or distort the near-end speech
• Therefore it is crucial to disable the NLP during
  Double Talk
• Timely and reliable detection of Double Talk
  could be tricky

17
Echo Canceller Factors

• Background noise continuity (level, spectrum)
  should be maintained
• The variety of real life echo paths poses an
  algorithmic and testing challenge
• field experience crucial
• EC standards (e.g. G.168) specify testing
  methods - not standard algorithms
• proprietary EC algorithms could vary
  significantly in quality
• Quality testing is essential

18
Echo Canceller Tradeoffs

• Attenuation vs. tail length vs. adaptation speed
  vs. steady state coefficients noise and EC
  stability
• Echo Attenuation in Single Talk vs. Double Talk
  performance
• Complexity ( MIPS)

19
Packet Loss Concealment

• Vocoder type (CELP vs. ADPCM vs. PCM)
• All vocoders (including G.711) have PLC
  standardized to some extent
• Vendor distinction depends on Jitter Buffer
  quality and PLC enhancements
• Trade offs
• Latency vs. Jitter buffer size vs. frame
  erasure rate
• Redundancy schemes vs. bit rate

20
Condition Interaction Effect

• Effect
• Customers tell us that G.723.1 many times sounds
  better than others G.711
• Why?
• Lower bit rate reduces congestion and results in
  fewer dropped packets.
• G.723.1 also includes packet loss concealment,
  smoothing over the gaps.

21
Improving VoIP Quality

• Techniques vendors use to maximize quality
• Dynamic Jitter Buffer
• Adjusting to ever-changing network conditions
• Packet loss concealment
• All coders, including G.711
• Echo cancellation algorithms
• VAD and CNG algorithms
• Extensive field and lab testing

22
Speech Quality Testing

• Methods
• Events
• Results

23
Speech Quality Testing Methods

• MOS
• Traditional subjective method, uses a panel of
  listeners, accurate but expensive
• PESQ - Perceptual Evaluation of Speech Quality
• objective method based on human hearing model,
  ITU P.862 standard
• Computer driven
• G.168
• ITU standard for echo canceller testing, uses
  artificial speech signals
• P.501 / P.502
• ITU standard for complex test signals used to
  test echo cancellers and other modules.
• These signals were designed to provide more
  information to the designer about the weakness
  points.

24
ETSI Speech Quality Test Event

• Sponsored by ETSI - Done by HEAD Acoustics Labs
• Has been held several times over the past three
  years
• VoIP equipment testing in controlled conditions
• Receive an objective and comparative analysis
• The tests include a range of aspects, including
• Speech sound quality
• Echo measurements
• Double Talk performance
• Transmission quality in the presence of
  background noise
• Anonymous - Only your results are identified

25
ETSI Test Setup
26
ETSI results Pie Chart

• Six conversational and six listening tests
• Ten gateways participated in 3rd SQTE
• The results for each gateway are summarized in a
  Pie Chart with a slice per each category
• The performance in each category was represented
  by the radius of the slice and by a color code
• Red the amount of which the results were under
  the requirement
• Yellow result was below the requirement if
  accompanied with red or OK if no red
• Green result was above the requirement

27
ETSI SQTE Test Results
28
ETSI SQTE Test Results
AudioCodes
29
How can you use this information?

• Ask for ETSI results when shopping
• Read and evaluate performance needs against test
  results.
• Justify partner selection
• Help sell your solution

30
Where to learn more

• Full ETSI SQTE Test Report
• http//www.etsi.org/plugtests/History/2004SQTE.htm
  
• AudioCodes ETSI SQTE Results
• http//www.audiocodes.com/Content.aspx?voip2189

31

• Lets talk! Stop by Booth 405