Performance testing Execution approach

Performance testing approach:

Before deployment to production, at least three cycles of load tests followed by a stress test and endurance test (at least for 12 hours) will be planned. Baseline and benchmark testing will be performed before load, stress, and endurance testing.

Approach

Steps 1 – Baseline Test: For the application, irrespective of time availability, performance testing will start with baseline testing. Baseline testing is nothing but one user test, which will help to identify the correctness of the test scripts and check whether the application meets the SLAs (Service Level Agreements) for a one-user load. These values can be used a benchmark for newer versions to compare performance improvements.

Step 2 – Benchmark Test: Next is the benchmark test for at least 15–20% of the target load. Benchmark testing helps to identify the correctness of the test scripts and tests the readiness of the system before running target load tests.

Step 3 – Load Test: Always plan to run at least three rounds of load test. Despite doing a slow ramp-up, it’s advisable to have three individual load tests for 50%, 75%, and 100% of the target load. (The load level should be defined based on system performance rather than just 50%, 75%, and 100% target load levels.)

Test Scenario – Have a slow ramp-up, followed by a stable period for at least an hour, and then ramp down. During the stable period, the target user load needs to perform various operations on the system with realistic think times. All the metrics measured should correspond only to the stable period and not the ramp-up/ramp-down period. Transaction response time should not be concluded based on only one or two iterations. The server should be monitored for a minimum of five iterations at the same load level before concluding the response time metrics because there could be some reason for higher/lower response time at any given time. That is why it is advisable to watch server performance for at least five iterations at the same load level (during stable load period) and use the 90th percentile response time to report the response time metrics.

Step 4 – Stress Test: Load tests should be always followed by stress tests. Based on the load test results, slowly increase the server load step by step to determine the server break point. For this test, realistic think time settings and cache settings are not required, as the objective is to know the server break point and how it fails.

Step 5 – Endurance Test: Endurance (stability) tests need to be run at least for 10–12 hours to identify the memory bottlenecks. They need not be run for peak load but it can be run for average load levels.