When financial institutions seek end-to-end testing options, they hope to ensure infallible, reliable payment systems operation. However, end-to-end test plans typically do not include performance or stress testing. Why not? Meaningful performance and stress testing of payment systems is perceived as expensive, complex and difficult.
Consequently, it often drops off the testing schedule. But the significant changes that financial institutions face today demand definitive testing to ensure system performance, reliability and recoverability. In our white paper, How End-to-End Payment Systems Testing Really Ends: Evaluating Performance, Reliability and Recoverability, we further address how to incorporate performance and stress testing into the test plan.
Even though you may agree that your organization needs to conduct performance and stress testing, you may not know how to begin preparing for these tests. Key considerations for planning for this type of testing should include the following:
Access to a knowledgeable team
Assembling the right team will do as much to ensure the success of your performance or stress test plans as any other factor. When developing a test plan, you must gather information and ensure cooperation from many others in your organization such as database administrators, systems engineers, network administrators, etc. Getting input from these team members is a critical first step in successful performance or stress testing, because your team members can provide valuable suggestions about what and how to test.
Clear goals and objectives
It’s impossible to know if your testing was successful if you don’t have clear goals before testing begins. A critical step in identifying those goals is defining your project scope.
In performance testing, for example, are you attempting to fine-tune an application, system or network? Are all your system end-points (devices, interchanges, etc.) required for testing? Performance testing can be as specific as having your developers use a profiler to test a particular routine, modify it and retest it to see if processing time in that routine improves. Regardless of the focus of your performance testing, you must target resource usage thresholds and processing times so you can use those during the test and when evaluating the test results.
In stress testing, for example, are you testing in regard to capacity planning—testing your ability to handle increased transactions due to growth, acquisitions or anticipated holiday/seasonal peak periods? Stress testing should determine the breaking point of your system as well as examine the usefulness of messages associated with system failures and the ability to recover data associated with the transactions that could not be processed.
Specific requirements for the test environment
Typically, your stress or performance test plan will include details about the test environment. Obviously, testing requires a test system representative of your production system and the availability of the proper personnel and tools.
The “last mile” of testing doesn’t need to be the hardest. After you have a test plan with clearly defined goals, you can evaluate tools by measuring their usefulness in achieving those goals. Make certain your end-to-end test solution includes performance and stress testing tools that can mimic all the end-points in your system and deliver meaningful results on system performance, reliability, and recoverability. For more information, please download our white paper, How End-to-End Payment Systems Testing Really Ends: Evaluating Performance, Reliability and Recoverability.