The Edge Blog

Banks In Need of New ATM Testing Practices

Blog Single

The situation sounds comical if you haven’t lived through it. Intentionally taping bills together to check how the cash slot will react. Cracking ATM cards before feeding them into machines. Wandering to various locations to repeat the process. Sadly, this is the legacy manual routine employed by many banks in testing the rollout of new ATM software and hardware.

Not only do these hacks risk permanent damage to ATM units, but they reflect an incomplete ATM network testing regime by definition. In many cases, ATM testers have at least been spared the indignity of roaming public streets, a stockpile of cash and cards in hand, looking for a variety of machines. One major bank houses 65 separate ATMs in its test lab, yet still falls well short of replicating all permutations. Another institution maintains 35 discrete ATM models but has floor space for only ten, leading to the bizarre sight of forklifts traversing the space to swap out units.

Falling Well Short of the 80/20 Rule

When limited to manual ATM test routines, banks by default begin to make subjective decisions regarding which scenarios to test. Most settle on test cases numbering in the hundreds; we’ve seen a few thorough FIs extend that to one or two thousand. Unfortunately, this leaves most banks far short of the well-known “80/20 rule.” By our math, most bank configurations encompass 5,000-10,000 ATM test cases. In other words, 10-20% testing coverage is common. For institutions running identical versions and components across its entire ATM network, the gap isn’t nearly as stark. But such FIs are the exception rather than the rule – consider the various transaction and card types today’s ATMs must process, and it’s easy to see how quickly the number of scenarios can propagate.

Obviously there’s a better way. Paragon Virtual ATM provides a virtual testing environment that simulates a bank’s ATM hardware network and transactions, and automates regression testing, opening the process to testers who lack access to the bank’s physical ATMs.  This approach allows validation of the entire ATM software stack through the XFS layer, making accessible XFS commands and messages that were not previously visible. The physical machines are shielded from harm, and cardholder experience is optimized. Best of all, ATM testers are freed up for higher value-added tasks like creating new test scripts – and with routines run overnight via automation – the testing of as many cases as can be devised.

Downtime Equals Downside

Given this exposure, the ATM industry has been very fortunate to have lived through a highly stable period since Windows replaced OS2 as the underlying operating system. While most development shops have been refined into factories, ATM system development has remained largely a “cottage industry,” relying on long tenured employees with deep personal domain knowledge to assess the risk of changes. Newer development techniques like Agile have yet to take hold.

System-wide ATM outages have been rare, largely because such failures are usually tied to the ATM switch, for which testing routines are better established. By contrast, individual ATM outages tend to be driven by local software. Every percentage point of overall system uptime can translate into hundreds of thousands of dollars in profit for operators of large networks.  In major metro areas, an out-of-service ATM may be met with a shrug and a walk to the next street corner. Of course that street corner likely belongs to another FI, meaning a lost revenue opportunity and potentially a ding to customer satisfaction/brand perception.

The Era of ATM Stability is Over

The grace period enjoyed by ATM operators is rapidly coming to an end, however. The current US EMV deadlines – which will officially begin to impact US ATMs this September – will have ripple effects through the entire system, but is merely the tip of the iceberg. Microsoft has announced plans for a full migration to Windows 10 to be completed by 2020 starting the sunset clock ticking for the ongoing support of Windows 7. This change will touch virtually every aspect of the ATM system – and don’t be lulled into a false sense of security by the seemingly far-off date; recall how quickly those EMV deadlines rolled around.

Also looming on the horizon is the end-of-life status of BASE24 and CONNEX – the two dominant ATM switch products. Even if they are not fully sunset, banks can expect hefty increases in related maintenance fees to provide a subtle nudge toward migration.

Let’s face it – ATMs are not sexy. Their low profile complicates the budget ask for sophisticated testing tools and other major overhauls. But the risk of remaining mired in limited manual ATM testing routines is about to increase dramatically.

Call us today at +1 919.567.9890 or Email us at

Get In Touch