Today, you can’t afford to keep performance as an afterthought as the cost of any performance issues in production are too high. It is mainly due to the rising user expectations. In a study by Akamai, 22% of the users choose not to return to the application when the performance didn’t match their expectations. Even a few seconds of delay is enough to affect your bottom line. If performance is so crucial, how do you test it at an Agile velocity?

You need continuous performance testing to deliver high-performing applications at speed. Simply put, continuous performance testing is to certify that every build meets certain performance standards and doesn’t affect the overall performance of the production. For this, continuous performance testing becomes an important step in the Continuous Integration (CI) pipeline. As developers constantly write new features and update the application, continuous performance testing gives early feedback on potential risks.

It may seem pretty straightforward but continuous performance testing comes with a few challenges. The testing throughput can be low due to the complexity of the application. This problem is compounded when skilled resources for performance testing are scarce. Moreover, you cannot exactly replicate the production environment. And even if you try replicating, the costs may not be justified. However, you can bring efficiency with the following practices:

Real-World risk analysis

You must account for the user habits and how they interact with your application. Knowing criticality of a user workflow helps in avoiding performance issues that have a deep impact on the user experience. For example, an ‘Add to Cart’ button is critical for the shopping experience on an e-commerce application. So, a risk analysis of all such user navigations can be carried out to detail and prioritize the test cases.

Maintain test suites

Test fragility has been a consistent problem in performance testing as a lot of time is wasted in fixing the test scripts. While many tool providers promise to solve this issue, they typically provide a quick fix rather than a sustainable solution. Testers will need to put a consistent effort to follow best practices in test maintenance.

Automate everything

Continuous performance testing is automation in a meaningful manner which is more than a request-response on a single point. Performance tests are executed on each component and various stacks. However, a reliable infrastructure is imperative to cut down waiting times and move things faster in the CI/CD pipeline. There are many cloud-based tools that can be leveraged to have seamless automation. But you need to find the right set of tools and build the infrastructure for optimal efficiency.

Track and benchmark performance metrics

Keeping historical performance results will help in visualizing performance trends but the analysis of metrics could be tricky due to the lack of proper tools. It is critical that you know the Key Performance Indicators (KPIs) for your application and set a benchmark metric that allows testers to evaluate whether to pass or fail a test.

Proactive monitoring

As replicating a real-world production environment can be infeasible, the performance of the application can be proactively monitored using synthetic monitoring tools. Synthetic monitoring uses various technologies, including bots to mimic user behaviour in various production environments. It can give preventive feedback on performance risks related to different browsers, devices, operating systems and geographical locations. Any issues can be resolved before it affects users. Hence, it complements continuous performance testing to ensure a quality digital experience.

In summary

Performance testers need to understand the business risks associated with performance issues. An end user perspective helps identify risks and prioritize tests. A prerequisite for continuous performance testing process is good maintenance of test cases, a seamless automation flow and performance metrics analysis. However, continuous performance testing has the limitation in replicating real-world environments that can be overcome by proactive monitoring.

Quality engineering is more than tools and tweaks. At Pyramid, we believe in building sustainable processes to deliver the best digital experience. We have more than 20 years of experience in consulting for successful IT delivery. Connect with our team to know what we can do for you.

By Raj Kasu October 23, 2018
Tags: Continuous TestingSoftware testing