The primary goal of load testing is to ensure that the Citrix environment can handle real-world conditions. By simulating the behavior of multiple users, applications, and data volumes, load testing aims to:
- Identify and correct issues that arise from simulating “live” conditions.
- Determine the optimal working capacity of the system and its components.
- Assess the working capacity of various applications when used concurrently.
The most important aspect of load testing is creating a realistic simulation of how the environment will be used by actual users. It’s essential to replicate real-world conditions as accurately as possible to identify potential issues before the system goes into production.
Load Testing: Extending Performance Testing
Load testing is an extension of performance testing, which establishes a baseline for system performance under normal operating conditions. Once this baseline is set, load testing pushes the system beyond its typical usage to simulate heavy workloads. This involves:
- Increasing the number of concurrent users.
- Increasing the volume of data being processed.
- Increasing the duration of the tests.
By testing the system under these conditions, any deviations from the performance baseline can be identified. Minor deviations may be acceptable, but significant performance drops signal serious issues that need to be addressed before the system is put into production.
More Than Just Numbers: Load Testing Is About Stability
Many see load testing as a “numbers game,” where the focus is solely on how many users a Citrix server can handle without performance degradation.
For instance, you may hear statements like, “We managed to get 45 users on the server!” While these metrics provide some insight, load testing is much more than just determining how many users a server can support. The primary focus should be on ensuring the usability and stability of the system when placed under realistic, heavy loads.
Example: Transaction Times Under Load
Consider a scenario where performance testing established that it takes about 2 seconds to submit a form and populate a database when using an application installed on a Citrix server. The data being entered includes both fixed data (such as names and addresses) and variable data (such as free-form text).
When 60 users are working concurrently, the database update time remains constant at 2 seconds. However, as the number of users increases beyond 60, performance starts to degrade:
- 61 to 65 users: 16 seconds per transaction.
- 66 to 75 users: 20 seconds per transaction.
- 76 to 80 users: 32 seconds per transaction.
Would it be acceptable to support 60+ users if transaction times increase eightfold? This is just one transaction type, and real-world users perform hundreds of transactions daily. Such a drastic performance drop would severely impact user productivity.
Based on this data, we might conclude that the Citrix server can safely support 60 users. However, further testing—particularly focused on the impact of variable data—would be necessary to validate this assumption.
Transaction Timings: A Key Indicator of Usability
Load testing helps determine how many users a Citrix system can support efficiently by examining transaction timings. Performance testing gives us baseline data, but longer test durations are crucial to uncover potential resource utilization issues, such as memory leaks. These issues often appear later in the Citrix server’s operating cycle (i.e., the time between server reboots).
For example, let’s look at how transaction times evolve over five days of continuous operation with 60 users:
Day 1:
- 1 to 60 users: 2 seconds per transaction.
Day 2:
- 1 to 60 users: 2 seconds per transaction.
Day 3:
- 1 to 60 users: 2 seconds per transaction.
Day 4:
- 1 to 42 users: 2 seconds per transaction.
- 43 to 55 users: 3 seconds per transaction.
- 56 to 60 users: 6 seconds per transaction.
Day 5:
- 1 to 38 users: 2 seconds per transaction.
- 39 to 51 users: 5 seconds per transaction.
- 52 to 57 users: 9 seconds per transaction.
- 58 to 60 users: 16 seconds per transaction.
On Day 5, we see that the Citrix server’s performance begins to degrade significantly after 38 users. Transaction times grow unacceptably high, with 60 users experiencing an eightfold increase in transaction time. Based on this data, we could conclude that the system can only reliably support 38 users.
However, there are potential strategies to mitigate this:
- Reduce the Citrix server’s operating cycle to 3 days, allowing up to 60 users to work concurrently without significant performance degradation.
- Adjust the server reboot schedule to maintain optimal performance.
Extended Testing for Realistic Results
Running load tests over an extended period allows you to accurately identify resource utilization issues that would not surface during short tests. These issues, such as memory leaks or CPU bottlenecks, often worsen over time and may only become apparent after days of continuous operation.
In the worst-case scenario, as the Citrix server’s resources become depleted (e.g., by Day 5 in the example), the system could stop responding entirely. If one Citrix server becomes unresponsive, the Citrix load balancer would attempt to redirect new user sessions to other servers. However, if those servers are also near their resource limits, they could also fail, leading to a cascading system failure across the entire Citrix site.
To prevent this, it’s crucial to run tests for as long as the Citrix system’s operating cycle—the time between server reboots. For example, if servers are rebooted once a week, the load test should simulate real usage conditions over that same period. Shorter test periods may not uncover long-term resource leakage issues, which can cause serious problems once the system goes live.
Resource Leakage: A Common Citrix Challenge
One of the key issues that load testing can uncover is resource leakage. In a Citrix environment, applications are not always optimized for multi-user setups. A memory leak that might only consume 20MB on a desktop PC could be magnified across dozens of users in a Citrix environment, resulting in hundreds of megabytes of lost memory.
In earlier systems, such as Windows NT Terminal Server Edition (which Citrix ran on), memory leaks were a common problem, often necessitating daily reboots. Today’s Windows Server operating systems are much more stable, and typically only require weekly reboots. However, resource leakage remains a potential issue, and thorough load testing can help identify when system reboots are necessary to maintain optimal performance.
Conclusion: The Importance of a Clean System for Load Testing
A clean and stable system is essential for accurate Citrix load testing. By simulating real-world conditions over an extended period, load testing not only helps identify potential performance issues but also guides decisions on server capacity, operating cycles, and reboot schedules. This proactive approach reduces the likelihood of system failures or severe performance degradation once the system goes live.
Rather than simply focusing on how many users a server can handle, load testing should simulate the entire operating cycle of the Citrix environment to uncover resource utilization issues that may impact performance over time. This will help ensure that your Citrix infrastructure is capable of meeting the demands of real-world users without sacrificing stability or usability.