How End-to-End Payment Systems Testing Really Ends: Evaluating Performance, Reliability, and Recoverability

Do end-to-end test plans end too soon? Perhaps—if they do not include meaningful performance and stress testing. Perceived as expensive and difficult to execute, performance testing and stress testing are often eliminated from the test schedule in favor of other test options. However, as institutions face significant changes in systems, product offerings, and delivery channels; the ability to unequivocally determine system capacity, reliability, and recoverability is essential. This article illustrates that this "last mile" of payment systems testing doesn't need to be the hardest.

When financial institutions seek end-to-end testing options, they hope to ensure infallible, reliable payment systems operation. However, end-to-end test plans typically do not include performance or stress testing—testing that is arguably the final step in guaranteeing solid payment systems operation. Why? Meaningful performance and stress testing of payment systems is perceived as expensive, complex, and difficult. Consequently, it often drops off the testing schedule. But the significant changes that financial institutions face today demand definitive testing to ensure system performance, reliability, and recoverability. This article relates financial industry trends with the system changes likely to result from them, and offers suggestions for planning to incorporate performance testing and stress testing into your organization's end-to-end testing.

Testing System Performance, Reliability, and Recoverability

The terms "stress testing" and "performance testing" are frequently used interchangeably. In fact, these types of testing are conducted differently and each has different goals.

• Performance testing determines the speed at which a system can process a specific load and allows the tester to observe the effect on the system’s speed and stability as resources are taxed. This information is then used to determine where and how to tune the system. Performance testing can also be used to determine system capacity; specifically, the point at which demands on the system can degrade response time or result in unpredictable or erroneous processing. The goals of performance testing are to eliminate bottlenecks and improve system reliability.
• Stress testing is designed to stress the system to failure. Stress testing investigates whether the system crashes, or fails gracefully (i.e., provides meaningful error messages) and recovers from failure without critical data loss. (Because stress testing takes the system to failure, disaster recovery scenarios can also be tested to ensure that a financial institution can rely on their disaster recovery processes to take over processing in the event of the primary system’s failure.) The goal of stress testing is increased recoverability.

Both types of testing are important. This article addresses the need for and requirements of performance testing and stress testing of today’s payment systems.

Copyright © 1996-2010, Paragon Application Systems