Method in the Madness: A Case for Virtual ATM Testing

Posted by Steve Gilde on January 15, 2018

Many organizations have adopted or have at least started moving toward Agile development practices abandoning the Waterfall methodology used for so many years. The “sprint” approach utilized in Agile development allows organizations to improve time to market for new features and provides a means to accommodate frequent updates to operating systems and regulatory mandates. Some development shops have evolved into decentralized operations, with developers, testers and QA resources often working remotely (including in offshore locations) as opposed to being huddled together in a single facility. These approaches have been undertaken to cope with the rate of change that has resulted from the need to accommodate the increase pace of innovation, the drive for cost containment, and the ongoing pace of mergers and acquisitions. 

In many cases, testing standards have not kept pace with this evolution of development principles. Many legacy testing processes remain centered on the desktop workstation, which results in many senior testers needing to spend more time running tests than designing them. This drag on productivity is further exacerbated by the fact that this type of testing methodology requires access to a model of the ATM-type being tested; ultimately all tests must be run on a physical ATM. This limits an organization’s ability to utilize offshore resources, limits collaboration and requires a test pen full of each model of the ATMs an operator wishes to test. 

The variety of ATM functionality can further compromise efficiencies when using traditional desktop ATM testing tools. For some segments of society, the appetite for cash can be effectively served by the barebones “cash and dash” ATM models often found at convenience stores. On the other end of the spectrum are the customers accustomed to the highly sophisticated, full-service machines (offering bill payment and other features).

Technology Surpassing Traditional Testing

Innovations like “cardless cash” are rapidly taking hold in the US. Given the introduction of Apple Pay and other mobile phone-based digital wallets, it logically follows that many consumers will expect the ability to similarly access cash using their phone as opposed to the traditional plastic card. This carries implications for both ATM software and hardware and extends to other logistical issues such as the ability of the consumer to unlock the unattended doors behind which many ATMs sit.

These factors may be why ATM testing is a more specialized field than much of traditional IT testing, requiring knowledge of a wide array of touch points and interdependencies, both internal and external. This limits the number of resources available to help an organization protect its brand and reputation at the ATM. That is why many organizations with legacy test programs too often use the 80/20 rule.  This approach, which is not limited to ATM testing, focuses on the 20 percent of use case scenarios most likely to expose 80 percent of the flaws.

Even if this “rule” is applied accurately, it leaves the organization counting on it with a significant level of exposure. In most cases, however, our experience suggests that ATM network operators, along with other players in the financial services industry, generate far less use case coverage than they realize, often well below 20 percent. No one who is head of an ATM testing organization could sleep well knowing this.

And after all, it only takes one wayward scenario to cause a fault, which could cause a handful of transactions to be lost or create a network-wide outage.  This means that just because a network has not been down for years, that it won’t be down tomorrow. Even if the person whose career rests on providing the appropriate level of testing considers this a good bet, successful gamblers always take steps to improve their odds. Thus, any organization can benefit from testing as many scenarios as possible, minimizing the window for unnoticed defects. Virtualization provides the opportunity to drive that number down much, much closer to zero than most organizations realize is possible.

Virtualization Improves Overall Testing

Virtualized testing solutions provide a repository for test cases and related media that is accessible, as required, by the members of a testing organization. As the test cases within the repository grow over time, the more benefit it provides decreasing the need to recreate tests scenarios and increasing the level of accuracy by allowing users access to previous tests in order to more accurately design new ones. In addition to these cataloged scenarios, virtual testing platforms store input from users across multiple installations providing an organization with the benefit of lessons learned by broader community of testers using the solution.

Concerning issues of compliance, it is a common misconception to believe that decreasing regulatory oversight will alleviate the strain on testing teams through reduced mandates. However, testing is usually done to determine if a coming change will impact the hardware or software elements within the system which the mandate will impact. Therefore, the rollback of a regulation will likely lead to the need for testing too since it, most likely, will create an operating change of some kind. Another “Catch-22” relates to the call by many organizations to decrease regulatory oversight so that IT resources can be freed to explore more areas where innovation is needed. More innovation, while good, will require more testing and better testing methodology to keep pace with technological and consumer demands.

Learn More About ATM Testing

Learn More About VirtualATM

Topics: ATM Testing, Agile, Virtualization