Integrating performance testing into continuous integration loops

Abstract. Performance testing is a practice, a technique, a process used for testing the speed, stability, scalability and responsiveness of applications. In contrast to the usual approaches
— that apply performance testing after functional testing and concentrate only on system-level tests — the authors discuss from a new perspective why and how performance testing can be integrated with other testing challenges (like functionality, security and usability), why and how it can be applied at all test levels in the life-cycle.
The aim of the authors is twofold: (1) to revisit the evolution of performance testing and to clarify the sometimes contradictory glossaries, (2) to add a guideline on how performance testing can be embedded into the software testing life-cycle in all test levels and how they interact with various testing goals in order to provide faster feedback, which is crucial in methodologies that apply frequent deliveries (like Agile, Continuous Delivery, DevOps). The findings are illustrated with practical examples from telecommunications, web-related applications and other problem domains.