Continuous Performance Engineering for Agile Teams
Perfana is a software solution that enables software development teams to regularly and quickly check their own software for performance issues, before going to production. Continuous Integration and Continuous Delivery need Continuous Performance Engineering.
Software Performance is Business Performance
Software Performance is one of the most important non-functional requirements for software. The impact of software performance on business performance is enormous and has been extensively researched and proven by the likes of Google, Amazon, Microsoft, and many others.
Poor software performance leads to a bad user experience, which may result in lost sales and low customer satisfaction. At the same time, bad software performance can have a significant impact on the bottom line of your business.
In today’s competitive market, where users expect more from your software than ever before, performance is no longer a nice-to-have feature — it’s a must-have. In short: Software Performance = Business Performance.
Continuous Performance Testing: Shift-left
If performance is that important, it seems evident that you should apply the same approach to performance testing as was introduced years ago for unit- and integration testing: automate it as much as possible and execute it as early and as often as possible. This minimizes the delay in feedback and allows the team to fix errors when the costs and the risk are still low.
However, the speed and scale at which software is being developed today is tremendous. Just a few years ago, a release frequency of several months to even a year was the norm. Nowadays, thanks to agile software development methods and DevOps, it is common for a team to release the software and put it into production at least once every two weeks. For the top-performing DevOps teams, this frequency is even multiple times per week or even multiple times per day.
In addition, software is no longer developed by a single programmer hacking away in their attic. Software development today is often an incredibly complex operation involving hundreds of components (applications, websites, services, apps, modules, etc.) created by dozens (up to hundreds) of teams. Those teams need to collaborate to make all those parts fit together and in-addition be well-tuned in terms of performance.
To achieve this scale and speed, software testing (technical and functional) is automated to the greatest extent possible and “shifted to the left” (shift left) to minimize the feedback loop. Shift left means that testing is done as soon as possible after the code is written.
Performance Testing at the speed of DevOps
Perfana manages the execution of tests from within the CI/CD pipeline (using 3rd party load generators) and handles the collection, visualization, and analysis of data, as well as the validation of requirements and reporting on them. So far, not much different from other performance test solutions. However, there is a difference.
There are many performance test tools that claim to make the life of a performance tester much easier. And we agree. They indeed make sense from the point of view of a single performance tester. Just as in the 1990s modern-day IDEs made the life of individual developers much easier compared to Notepad or vim. However, how does that work at the speed and scale today? When you are working in a team that needs to work with other teams and you want to release every week or every day? That’s where Continuous Integration and automated testing came into the picture. This enabled automated testing and integration in order to be able to scale and speed up.
What really sets Perfana apart from other solutions is that most focus on load generation. Perfana focuses on automating the test run result analysis. By automating this labor intensive and time-consuming task it enables developers to get constant feedback on the performance of their software, without claiming significant effort.
This is what Perfana brings to performance testing: at scale, continuously, without manual intervention unless a requirement is not met. We believe this is the only way you can validate the performance of software in a modern, agile software development operation before going into production.