Double Duty: Quality Voice without Data Starvation

By Lisa Phifer

September 25, 2007

Report benchmarks Meru AP voice quality & airtime fairness for converged traffic in competitive test against Cisco.

As enterprises deploy VoIP, vendors are being challenged to provide toll-quality voice and predictable data delivery using shared WLAN infrastructure.  While prioritization helps, WLAN architecture and AP engineering can also make a huge difference.

 

According to Paul Curto, Senior Technical Marketing Engineer at Meru Networks, “For customers who are deciding amongst enterprise WLAN vendors, the best test is one that incorporates real-world clients, making independent over-the-air decisions.”

 

To facilitate this, Meru developed tests enabling objective measurement and subjective assessment of voice quality, airtime fairness, and aggregate throughput for simultaneous voice and data streams run through a single AP.

 

“Our customers are making choices between competing vendors.  We wanted to show that we have the best solution for converged enterprise voice and data,” said Curto.  “We looked to The Tolly Group to provide third party validation and to create a test recipe that customers can run for themselves to see how our solution scales.”

 

Testing voice quality

 

According to a new report published this week by Tolly, engineers put Meru and Cisco WLAN solutions through a pair of benchmark tests in an RF-controlled testbed.

 

One test measured VoIP quality and aggregate throughput for an offered load consisting of 6 live VoIP calls, 22 simulated VoIP sessions, and 8 data sessions.  Simulated VoIP streams were generated by IxChariot, using a 30 ms payload and G.711 codec to consume roughly 192 Kbps each.  Data sessions were also generated by IxChariot, using 1416 byte buffers to send traffic as quickly as the WLAN allowed.

 

Running through a Meru dual-radio 802.11g AP 208 and MC 3000 controller, these streams produced a combined throughput of 4 Mbps.  As measured by IxChariot, voice quality averaged 4.07 MOS (Mean Opinion Score) – anything above 4 is considered toll quality.  Voice packets also averaged 17 ms of latency and 2.7 ms of jitter – well under the 150 ms acceptable delay threshold defined by G.114.

 

To complement these quantitative numbers, engineers also placed live calls, using half a dozen Ascom i75 Wi-Fi phones.  In their subjective opinion, Meru calls resulted in “good sound quality” at six phones and “did not experience noticeable degradation” at eight phones.

 

Vying for air time

 

In a WLAN without prioritization, data traffic can significantly degrade voice quality.  Conversely, even one voice call can have an impact on aggregate AP throughput, since VoIP keeps a shared channel busy by sending a large number of short packets.  Wi-Fi Multi Media (WMM) attempts to address these problems through prioritization.

 

Because was voice was given higher priority and testers sent data packets as fast as the WLAN would accept them, the data clients essentially shared the available capacity not consumed by voice.  Total throughput inched from 4.0 to 4.1 Mbps during a second test that increased the number of data clients from 20 to 30.

 

However, “WMM doesn’t do as good a job of managing traffic in the upstream direction,” said Curto.  “Meru supports WMM, but we don’t think WMM solves the whole problem.  Our Air Traffic Control ™ technology manages both upstream and downstream traffic.”

 

Specifically, in both tests, the number of upstream (client-to-AP) and downstream (AP-to-client) data transactions were identical.  “Every single flow had an equal and fair distribution in both directions,” said Curto.  “This airtime fairness is something that’s very important to our customers, not only for traditional data applications, but for other mission critical applications like video.”

 

According to Curto, by taking QoS guarantees a step further, Meru can offer a more scalable solution for converged voice and data networks, as well as stronger quality of service guarantees for voice.  “The single AP test in this report is a good representation of our capabilities at scale, but we have also conducted a pervasive all-wireless office demonstration [that] extends the same principles to the all wireless Enterprise,” said Curto.  Those results will be published in an upcoming Farpoint Group technical note.

 

Checking out the competition

 

The Tolly report compares Meru results to those generated by running the same traffic through a Cisco AP1242AG and 4402 WLAN controller.  According to the report, Cisco delivered half of Meru’s aggregate throughput, relatively poor 1.6 MOS call quality, an unacceptable 813 ms one-way latency, and 40 percent fewer upstream transactions than Meru.  Cisco did not respond to Tolly’s invitation to actively participate in the test plan or review/comment on test results.

 

“This test illustrates that Meru’s over-the-air Quality of Service can manage VoIP and data traffic in such a way as to deliver toll-quality voice in a multi-user high density enterprise environment while Cisco’s solution fails to deliver acceptable quality for any of the VoIP users,” said The Tolly Group’s founder Kevin Tolly.

 

As with all benchmarks, numbers generated in a clean lab environment are no substitute for in-situ testing with representative traffic and real-world scenarios.  For example, one important voice variable is the codec.  “Here we used a G.711 codec,” said Curto.  “If you were to use a lower quality codec, you’d have lower bandwidth consumption.”

 

Meru designed these tests to reflect traffic they have seen used in the field, with key metrics demonstrated through both subjective quality assessment and quantitative measurement.  However, by documenting test methodology and set-up, Curto hopes that “customers will run bakeoffs themselves, inviting other vendors to come in and run the same tests.”   The full report is freely available for download here.



Comment and Contribute
(Maximum characters: 1200). You have
characters left.