Traditionally, functional payment testing has been used to answer a simple question: did a transaction work as expected? Pretty straight forward from a developer or QA analyst perspective. A cardholder on the other hand would be asking not only did the transaction work i.e., was I able to get my money from an ATM, but what was the experience like?
So while a transaction that used to complete in 66 seconds that now takes 90 seconds after a software update may still pass a standard functional test, the difference in time standing at the ATM will certainly be noticed by your cardholder – as well as by every other consumer standing in the ATM queue.
Slow, but successful transactions rarely get logged as defects, which is why they so often reach production systems and have a negative impact on the customer experience.
One obvious reason for this gap is that it is difficult to keep detailed statistics on transaction activity when you are standing in front of an ATM with a stack of test cards, a clipboard and a stopwatch. While it is theoretically possible to manually capture accurate data on any one transaction, the probability is low that this can be done consistently over a large number of transactions, say a complete regression run.
On the other hand, VirtualATM automatically records the outcome and duration of every transaction. Run the same transaction 100 times and the platform precisely captures and calculates the timing of every test along with its average, minimum, and maximum duration, as well as its pass/fail rate. Within a few runs, your teams can easily establish a baseline for how long any specific transaction should take on any specific ATM configuration.
With this level of detail available and accessible, variations show up quickly, with any outliers easily investigated. For example, if a specific test runs significantly slower than expected after a software patch is made, you can simply select it for evaluation against your previously established baseline. VirtualATM displays a side by side comparison of the two tests, card insertion, PIN entry, host response, media dispense, receipt, etc., so you can review the transaction step by step and pinpoint the specific step(s) where any slowdown is occurring.
Instead of getting a vague feeling that the transaction ran slower after the patch, you know exactly how much slower it ran and where the delay is happening.
Most organizations run a mix of ATM hardware, application versions, and device configurations. VirtualATM supports NCR, Diebold Nixdorf, Hyosung, and other CEN/XFS-compliant ATMs through a single user interface, and gives teams several ways to compare the different stacks:
One thing that can be said about the ATM channel today is that it is dynamic. Self-service technology is continually evolving, security updates are never ending, screens changes, and transaction flows need adjustment as tests that once ran cleanly begin to fail. The VirtualATM Projects view can highlight the top failing script for each project so teams can focus their maintenance efforts specifically where it matters.
Screen comparison adds another layer of visibility. For example, when a captured screen doesn't match its baseline target, VirtualATM fails that step and reports the score, but keeps running to completion so that one pass reveals every mismatch so that they can be easily corrected.
Tracking this level of testing detail and having the information easily accessible to anyone who needs to see it provides ATM fleet operators with the tools to make faster, better business decisions. This not only helps keep the entire estate running in optimal condition, but delivers a number of additional benefits, such as:
Just because a test passes does not mean it is ready for production. VirtualATM provides teams with the data and analytical tools that need to quickly and easily measure and evaluate how well transactions really perform and then make the necessary adjustments to prevent customer satisfaction issues.
ATM performance data is only useful when it is available and accurate. VirtualATM runs your application software in a virtualized environment, with full CEN/XFS and EMV support across NCR, Diebold Nixdorf, Hyosung, and other XFS-compliant ATMs. That means the metrics you measure reflect precisely what your cardholders would experience when standing in front of one of your ATMs. Every run, step, and screen is recorded in the VirtualATM application database, so results and analytics are based on results that can be trusted.
Paragon resources will help ensure the success of your implementation, including set-up, configuration, and training, as well as assisting with your first projects and automated regression scripts. As your trusted ATM testing partner, we will assist you every step of the way to ensure that you get the results and reports you need to be successful.
Next in this series: turning testing data into a business case that executives can act on.
VirtualATM records the outcome and duration of every test transaction. Run the same transaction repeatedly and you get reports that document the pass/fail rate with average, minimum, and maximum duration. Within a few runs you have a baseline for that transaction on that ATM configuration.
VirtualATM allows users to select any transaction and compare it against your baseline (or any other transaction). You can review each step and determine exactly where variations occur.
The Projects view highlights the top failing script for each project (you can also get a view that shows every failing script with its fail count). The screen comparison feature also flags captured screens that don't match their baseline so that they can be quickly and easily adjusted.