Performance Test Environment Setup

Performance testing services are performed to check the behavior of the application under test (AUT) under different load levels and its post-crash recovery process. AUT performance results extensively depend upon the test environment. The first rule of thumb in performing a performance test is to have a test environment which is the exact replica of the production system as even the minor differences between the production and test environment can completely invalidate the entire test activity.

Complete knowledge of the AUT production environment, test environment isolation from other activities, network isolation, proxy removal from the network path, full access to the server, and warming up the server before testing are some of the approaches that are necessary in order to successfully execute a performance test.

Various test environment options are available with each having its own advantages and disadvantages and selection of a particular option depending upon the requirements. Following is the brief list of test environment options:

1)   A subset of the production system with fewer servers of lower specification

This is a very common test environment preparation approach where performance testing is conducted on an already established test environment regardless of its production environment specification. Although testing in such environment can help you identify some application bottlenecks, however, complete the application and its infrastructure will never be tested.

2)   A subset of the production system with fewer servers of the same specification

This is comparatively a quite better approach where testing is done on smaller application infrastructure but the results can be extrapolated to map them on full application scale. Although this environment produces better results, the post-testing bottlenecks can be missed very easily in the extrapolation process.

3)   An exact replica of the production system

This is the most desired test environment which is extremely difficult to achieve due to its complexities and financial constraints but if it is achieved, the test results are 100% guaranteed. Some of the challenges which the performance testing teams face while preparing the exact replica of the live system include:

• Replicating the production servers architecture

• Production system network infrastructure replication

• Replicating all application tiers of the production system

• Replicating the production system network bandwidth

• Replicating the production database with all its dataset

Some of the system components like the third party services, firewall impact, working of load balancers, application internet connection, and DNS lookup time, etc cannot be tested on the test environment.

4)   Testing on Production System

Another approach which is also largely practiced is to conduct the testing on the production system due to the concerns and complexities of the test systems. Although the best way to validate the system performance is to run the test directly on it, however, there are numerous concerns about this approach i.e. the system availability during test execution, its impact on actual users, protection and mixing of test and production data etc but if all these issues are handled successfully then the optimal performance test results can be achieved.

Furthermore, performance testing on a production system should be conducted based on its following advantages:

• Test results validation

• It’s cost-effective

• Helps in testing the database back-ups and post-crash recovery

• Easier to set-up the test infrastructure

The impact of testing on production system on actual users can be minimized by conducting performance testing during off-hours of off-days (i.e. around midnight during the weekends), during the maintenance window, before officially realizing the application and by performing read-only operations.

Performance Testing Production Environment 

There are various benefits and drawbacks of conducting a performance test in a production environment. Let’s have a look at a few of them before hiring a performance testing company:

Benefits:

• Testers do not require to reproduce the production site data set. 
• A reduction in cost and time involved in test infrastructure.
• It is possible to validate the performance test results that are conducted on the test environment.
• The application recovery process and its complexities are well-defined.

Drawbacks: 

• Real-time users experience slow applications and errors.
• Identifying bottlenecks in the presence of real users become difficult and tricky. 
• Databases may become slower after the test when a lot of data is generated. 
• It is most likely that testers may have to block access to real users in order to achieve the performance test results. 

Once testers have set up the performance testing environment, they can compare it with the production environment on the basis of various factors such as the number of servers, hardware and software devices, app components, etc. It is also important the tests are performed properly so that there are no issues pending in an app while ensuring its usage in real-life situations. 

Leveraging the advantage of our testing skill-set, we, at Kualitatem, offer load testing and performance testing services to ensure and evaluate the overall quality and performance of your software applications. Our team of adept performance engineers has the necessary knowledge needed to work with various testing tools and assist them in identifying and fixing all performance-based bottlenecks. Initially, we conduct the performance testing on our test environment and involve the development and network teams to fix all the identified defects. Once all the identified bottlenecks are fixed and verified in the test environment, we test the production system to validate the test environment results and test the missing components as well.