Load DynamiX enables scalability testing for QA team at F5 Networks

Summary

Challenges for F5The F5® team responsible for QA of the company’s BIG-IP® WAN acceleration technology has deployed Load DynamiX in their lab, cutting the time spent on testing in half and enabling the scaling of test scenarios to accurately simulate customer environments. No more writing homegrown tools or reconfiguring fragile scripts for each test case. The Load DynamiX 1G series generates CIFS traffic that simulates actual Windows®-based applications, with the flexibility to customize at will.

  • Manual testing tools. Time-consuming to develop and unwieldy to manage and maintain.
  • Lack of configurability. Difficult to configure CIFS traffic and little support for automation.
  • Limited scalability. Prohibitive to add enough servers to accurately test some customer scenarios.

Changes with Load DynamiX

  • Reduced testing time by 50%. Frees the team to do more valuable things; better use of talent.
  • Improved testing coverage. Scriptable and user-friendly with detailed protocol control and reporting.
  • Scalable for long-term growth. Real-world client- and server-side simulation; versatile, fast scalability with less hardware.

About F5 Networks

F5 provides the means for businesses to maximize IT infrastructure agility and effectiveness. F5 sells a variety of storage, networking, and application solutions that require their product teams to develop for the broadest imaginable range of customers. One of the significant capabilities of the F5 BIG-IP product line is WAN acceleration, including Microsoft® CIFS (Common Internet File System) protocol. And this testing lands in the lap of Terry Swartz, QA manager at F5. Terry is charged with implementing the most efficient test processes possible. As with all competitive businesses, he needs to do more with less each day. Terry says his goal is to automate as much as he can to relieve his team of labor-intensive, low-value tasks. So he is persistent in his hunt for the newest equipment and tools to do just that.

The Trouble with CIFS

The particular challenges faced by the F5 test team highlighted in this case study revolve around testing their WAN accelerator product capability for accelerating CIFS traffic. Dealing with CIFS presents a number of difficulties, including setting up your QA environment to accurately simulate Microsoft applications. This requires a deep understanding of the CIFS protocol. CIFS is a chatty protocol, generating a lot of traffic for every request. This chattiness provides an opportunity for companies like F5 to develop products that speed up file access.

F5 WAN Acceleration Basics

To greatly simplify the topology of the WAN acceleration testing configuration, one F5 BIG-IP appliance sits between the client machines and the network in a remote location. Another appliance sits in the datacenter between the network and the servers. This two-device testing setup leverages F5’s iSessions technology to secure and accelerate data traveling over the WAN. With iSessions, two BIG-IP devices can be deployed symmetrically to create a site-to-site secure connection to improve transfer rates, reduce bandwidth, and offload applications for more efficient WAN communication. The two appliances inspect CIFS traffic and respond to requests that are redundant or deemed unnecessary without sending them to the other end. This reduces the number of packets sent and improves overall network performance. To effectively test F5 BIG-IP’s WAN acceleration capabilities, the team must generate traffic on the client side that is Microsoft application specific, as well as accurately simulate application responses on the server side.

Before Load DynamiX

“Load DynamiX has provided a flexible feature set for effective testing. Its product enables us to create complex tests, scalable tests, and of course repeatable tests very efficiently through the user interface. And the automation capability plugs into our existing automation framework.”
— Terry Swartz, QA Manager, F5 Networks

The F5 QA team was manually setting up both client and server side, and writing test tools from scratch. Terry says they were writing a lot of complicated scripts and spending too much time configuring whitebox servers and the network. And with every new hardware platform and feature set that F5 delivers to its customers, the testing matrix gets bigger and the challenges compounded.

One Customer Scenario

A typical test for F5 is to simulate a customer with 200 employees using Windows desktops and sharing a filesystem. Testing a client load of a dozen desktops is easy, but to create realistic traffic for 200+ Windows clients is a daunting effort. And ideally, F5 would want to test a scenario with 1000 desktops. Such scale is practically impossible with manually-constructed testing environments, whether you’re using real servers or virtual machines. It’s no wonder that Terry Swartz is constantly on the look-out for tools and technologies to make his capabilities broader, his team faster, and results better.

Enter Load DynamiX

Load DynamiX offers a high-performance appliance featuring 8 ports that are each configurable as either a CIFS client or CIFS server. The Load DynamiX 1G series includes the Test Development Environment, a graphical application for configuring and running tests, and analyzing results. Terry provided invaluable input into the feature set and helped identify functionality critical to a company like F5. “Load DynamiX jumped all the hurdles,” says Terry, “they were responsive, receptive, and we got everything we asked for.”

Load DynamiX 1G Series at F5

F5 is using the Load DynamiX 1G series to simulate multiple CIFS clients and servers in a WAN acceleration test environment, with clients writing and reading F5-specific content to the servers. The machine is providing the automation Terry was looking for and has enabled him to reduce by half the manpower required to run the CIFS test. He has now moved engineering talent to more valuable tasks. And he’s realizing better results faster, with scaling potential for the long term.

The F5 Growth Path

For Terry’s QA team, the capacity to scale aligns with the company’s growth path. Instead of struggling with added requirements or playing catch-up with the company’s technology advances and increasing customer needs, they’re ready for whatever F5 developers come up with.

Isolation and Resolution

The Load DynamiX product isn’t only about traffic. The Test Development Environment software features a graphical interface providing granular configuration options and results analysis. Testers can now find problems, modify parameters, and quickly see results using the Load DynamiX interface. Problem isolation is fast and accurate. This helps Terry’s test team work extremely effectively with F5 engineers who can now address any potential issues more quickly.

Replacing Physical Machines

Terry’s mandate is to create efficient test configurations that can be used for as many scenarios as possible. Using a dozen whitebox servers to simulate average customer environments was one standard set-up. Those servers have now been replaced by a single Load DynamiX appliance that enables Terry to easily simulate 1000s of concurrent users. The bottom line is that the lab’s power limitations and maintenance costs make replacing racks of servers with a single appliance a huge win.

The Last Word

By using Load DynamiX, F5 not only realizes the physical benefits of eliminating hardware, they now reliably test application-level protocols in scalable, real-world environments, using fewer engineers with better results.

Load DynamiX 1G Series Advantages

  • Cost Savings: Costs less than the servers and software required to generate the same volume of traffic.
  • Real Content: User-specified content can be written to the server, read from the server, and verified.
  • Granular Control and Reporting: CIFS, NFS, and TCP/IP configuration options far exceed homegrown tools.
  • Quicker Test Cycles: Wizards and canned tests minimize test development time. Automation reduces test suite execution time.