Exinda gets results using Load DynamiX to test their WAN optimization systems

About Exinda

Exinda is a global supplier of next generation WAN optimization products, delivering solutions to over 2,000 organizations in 80 countries that reduce network operating costs and ensure consistent application performance over the WAN. The Exinda Unified Performance Management solution provides application visibility, control, optimization, and intelligent acceleration — all within a single network appliance that is affordable and easy to manage. Its main features help to optimize bandwidth utilization, set policies to control peer-to-peer and recreational Internet traffic, and ensure application performance over the WAN.

Exinda QA Challenges

Before Load DynamiX

  • Not enough connections. Need more users and more CIFS traffic to test real customer cases.
  • Not enough reporting. Can’t assess errors well enough to isolate and fix problems.
  • Costly integration testing. Tough to test interaction between tightly integrated product features.

After Load DynamiX

  • Bandwidth off the charts. More than enough connections with little overhead, no extra hardware.
  • Sophisticated reports. Real-time, detailed reporting for problem isolation and resolution.
  • Visible, measurable impacts. Easy to see the impact of each product feature on performance.

The network management functions of the Exinda appliance cover network optimization, application visibility, traffic control, and application acceleration. Exinda has separate licensing for different software offerings in one product, so QA must test components separately — QoS, application visibility, and acceleration — and together in a highly integrated environment.

Exinda provides the best of all worlds for its customers, with a high level of interaction between components. This tight integration is a key factor for testing, and presents a number of challenges for the Exinda QA team.

Exinda Test Environment

The test lab at Exinda supports product QA as well as development. The hardware setup includes dozens of cabinets of equipment—servers, product hardware, test equipment, and a variety of devices — all onsite next to the company’s development area. The lab is designed to need minimum intervention, with dedicated machines and very little configuration redo. It is automation heavy, with several racks devoted to automated setups. In addition, Exinda has dedicated testing environments for acceleration, requiring multiple appliances to emulate WANS, and simulate customer traffic patterns and topologies. John Bothe, chief architect for Exinda, tells us that QA includes a team of engineers who work closely with Exinda developers. They have resources dedicated to automation and some who are general staff, but all are responsible for test case development.

Exinda Testing Capabilities

With homegrown test tools, Exinda needed improved reporting, automation, and scalability. With a rapidly expanding customer base, their capabilities wouldn’t fully support the depth and breadth of customer use cases. John says that the test team’s main pain point was in testing CIFS, with two problems front and center. One, they needed to generate more TCP connections. Two, they didn’t have sophisticated, comprehensive test results reporting. Often, there was no indication of what might have gone wrong during the test, so getting to the root cause of the problems was costly. In addition, the interactions of Exinda product features were difficult and time consuming to test. John didn’t have sufficiently detailed reporting to measure the impact on throughput of various combinations of features, which led to longer manual test cycles.

Enter Load DynamiX

“We’re growing fast, and need test tools beyond what we can do inhouse. We found Load DynamiX to be better than any other solution we looked at and the Load DynamiX 1G series is more than filling our needs. Particularly with the depth and maturity of their CIFS tools, we can expand into new markets and support bigger customers.”
– Chris Siakos, CTO, Exinda

Exinda discovered Load DynamiX and quickly got an evaluation unit into the lab to put it through its paces. John says that their setup experience was good. The QA guys had a few questions for Load DynamiX, but a Web-based walkthrough beforehand enabled the team to use the interface and start testing on their own, out of the gate. In the buying decision, Exinda looked at other ways to solve their problems. With Load DynamiX, the cost per port wasn’t variable and there were no add-ons or licenses required. Exinda appreciated that the Load DynamiX 1G series was purpose-built and dedicated to high-capacity, performance-driven testing without hidden costs.

About the Load DynamiX 1G Series

A high-performance load-generating system, the Load DynamiX 1G series is designed for comprehensive functional and performance testing of IP storage. It provides protocol-level control and semantic definition that enables the broadest possible range of test cases. The product features 8 singly configurable 1GE ports, each capable of a million TCP connections. It also has a 1GE management port and supports multiple authentication types, originates NFS, CIFS, iSCSI, and HTTP traffic, and easily handles multiple test scenarios with varying lists of commands — all running concurrently. The product is the storage industry’s elite testing tool due to its storage-specific features, high-volume traffic handling, customer-centric graphic interface, and an automation framework.

Test Development Environment

“One of the biggest benefits is the ability to automate, with error detection and flagging. And, overall, for a single test case, we find at least 10x improvement — now 10-15 minutes compared to half a day or more.”
– John Bothe, Chief Architect, Exinda

Load DynamiX products include a graphical application, the Test Development Environment, that provides flexible, granular configuration options and test results analysis. Set up tests, find problems, modify parameters, and quickly see results via an intuitive drag-and-drop interface.

The Results

While Exinda’s homegrown tests did generate CIFS connections and do basic acceleration testing, the number of connections and throughput were limited and error reporting was weak, making troubleshooting difficult. John had servers and a console to run scripts driving one testbed at a time, but just one test at a time. With Load DynamiX, not only can John plug in and have CIFS traffic flowing in minutes, he can run multiple tests at once. He saves test settings, takes advantage of statistics and reporting, and accurately benchmarks against claims. For a single test case, what formerly took half a day or more — configuring servers and software to run the test, actually running it and then generating reports — now takes 10-15 minutes.

Beyond dramatic time savings, the single biggest benefit for the team is the ability to automate all of this and integrate high performance testing into their automation setups. For example, say you have continuous integration systems detecting changes to the source tree that kick off automated tests. Check that if you send 10MB, the box reports 10MB. Or if a normal 30-second sequence of traffic goes up to 60 seconds, it’s automatically flagged. Previously this was not at all simple to do with homegrown tools. In the Exinda lab, Load DynamiX 1G series ports are dedicated permanently to continuous automated testing. With rapid customer growth, Exinda will continue to add ports to scale with the ongoing expansion of the QA lab. The product has become an integral part of Exinda’s QA environment.

John is also using Load DyanmiX for general testing. Not only can they test CIFS, they can do a general verification of QoS under load. In addition, the product developers who work in the lab take advantage of the system to test feature sets and work in a more-accurately-simulated customer environment. All in all, Load DynamiX has made a significant difference in the productivity of the Exinda QA lab.