While load testing is a critical part of evaluating Citrix environments, it can be further expanded to include volume testing, which focuses on the amount of data or tasks each user performs while the system is under load. Volume testing ensures that not only can the system handle a certain number of users, but that it can also manage the intensity and complexity of tasks those users perform simultaneously.

Defining Load vs. Volume Testing

There is often confusion about the terms load testing and volume testing, with some organizations using the terms interchangeably. However, the distinction between the two is important:

• Load testing focuses on increasing the number of concurrent users on the system.
• Volume testing measures how the system handles increasing volumes of data or tasks that each user processes.

The key difference lies in what is being stressed—load testing measures user concurrency, while volume testing measures the complexity and number of tasks each user performs. Both are essential to understanding the full capabilities of a Citrix system.

Volume or Load? A Practical Example

An experience I had while working in banking illustrates the difference between load and volume testing. At the time, there was a major push to adopt blade technology for the Citrix site. I was tasked with designing a Citrix site to run development applications, but the site was set to be deployed using low-powered, non-SCSI, single CPU blades. Given the nature of the workload, I believed volume testing was crucial, but as an outsider and freelancer, the Citrix team did not initially share my concerns.

The prevailing belief was, “We should easily get 45 users per blade!” I was concerned because the figures being touted were optimistic and did not consider the complexity of the tasks that users would perform. I suspected they had overestimated the number of users that could be supported per server.

To demonstrate the importance of volume testing, I ran two distinct tests:

Test 1: Basic Load Test

In the first test, I ramped up the number of users on a single blade server and had each user open Microsoft Word. This was a basic load test where the goal was to see how many users could log in and perform simple tasks like opening a document. The results were impressive—180 users logged in, and the Citrix team was thrilled, as the results exceeded their expectations.

Test 2: Volume Test

For the second test, I shifted focus to volume testing. Instead of just opening a simple application, I had users log into a development application that ran various complex tasks, such as compiling programs. I worked with developers to script over 200 different tasks to simulate real-world usage. One of the applications involved in the test had over 100,000 lines of code, and I used this as the core of the volume test.

This time, the server struggled with just 15 users performing intensive tasks. Both servers used in the tests were identical in hardware specifications, but the Citrix team was surprised by the dramatic difference in performance between the load test and volume test.

The discrepancy became clear: while the first test involved light, terminal services-aware tasks (like using Microsoft Word), the second test involved high-volume, resource-intensive tasks. The second server was pushed to its limits because the volume of tasks being processed exceeded what the blade server could handle. This demonstrated how simply increasing the number of users does not always provide an accurate picture of system performance—the nature of the tasks matters.

Understanding Volume Testing Through Analogy

To further clarify the difference between load and volume testing, consider the analogy of testing a Multi-Purpose Vehicle (MPV). The MPV is designed to carry 7 passengers. In load testing, the vehicle would be tested with 7 passengers, driving across various terrains. This would assess how the vehicle performs under the intended load of passengers.

Volume testing, on the other hand, focuses on increasing the “volume” of the passengers without increasing their number. For example, instead of using average-weight passengers, you could test the MPV with obese passengers. While the number of passengers remains the same, the combined weight (or volume) of the passengers increases. This additional weight puts more strain on the vehicle, simulating real-world conditions where passengers or cargo may vary in weight.

Similarly, in volume testing for Citrix, we do not simply increase the number of users. Instead, we focus on the intensity and complexity of tasks each user is performing. This allows us to assess how the system handles increased resource demand without necessarily increasing the number of users.

Lessons from Volume Testing: Reassessing Hardware Choices

The results from the Citrix volume test prompted a reevaluation of the hardware being used for the Citrix deployment. The original plan to use blade servers, which were more affordable, proved inadequate when large volumes of tasks were executed. Despite the high user capacity in simple load tests, the blade servers struggled under volume testing, indicating that they were not a suitable choice for the system’s actual workload.

As a result, the decision was made to avoid blade servers and opt for dual CPU servers, which had previously been used in other deployments. These servers were better equipped to handle the high-volume tasks associated with development applications, ensuring better performance and stability.

Carrying More Data: The Impact of Volume on Performance

Volume testing is essential for understanding how systems respond to increased data loads. Just as adding heavier passengers strains an MPV, increasing the volume of tasks or data processed by users strains a Citrix server. Volume testing allows us to see how the system handles different intensities of resource usage, whether it’s processing large data sets, running complex applications, or handling multiple transactions.

In the case of Citrix, many applications are not designed with multi-user environments in mind. Applications built for standalone desktops may not handle multiple concurrent tasks efficiently in a shared environment. Memory leaks or CPU bottlenecks that are insignificant in desktop environments can become critical issues when amplified by the number of users and tasks on a Citrix server.

Conclusion: The Importance of Volume Testing in Citrix

While load testing is essential for determining how many users a Citrix server can support, volume testing provides a deeper understanding of how the system performs when users are executing complex or resource-intensive tasks. By combining both load and volume testing, organizations can gain a more accurate picture of their system’s true capabilities and limitations.

Volume testing goes beyond the numbers game and assesses the real-world conditions that a Citrix environment will face. As demonstrated by the banking example, hardware decisions, cost estimations, and system design must be based not only on how many users a server can support, but also on the complexity of tasks those users perform. By conducting thorough volume tests, organizations can ensure that their Citrix environment is prepared to handle both high user concurrency and heavy workloads without sacrificing performance or stability.