ATM Testing - April 24, 2024
Many organizations have adopted or have at least started moving toward Agile development practices abandoning the Waterfall methodology used for so many years. The “sprint” approach utilized in Agile development allows organizations to improve time to market for new features and provides a means of accommodating frequent operating system updates and regulatory mandates. Some development shops have evolved into decentralized operations, with developers, testers and QA resources often working remotely (including in offshore locations) as opposed to being huddled together in a single facility. An Agile approach helps organizations cope with the rate of change that has resulted from the increased pace of innovation, the drive for cost containment, and the ongoing pace of mergers and acquisitions.
In many cases, testing standards have not kept pace with this evolution of development principles. Many legacy testing processes remain centered on the desktop workstation, which results in many senior testers needing to spend more time running tests than designing them. This drag on productivity is further exacerbated by the fact that this type of testing methodology requires access to a model of the ATM-type being tested; ultimately all tests must be run on a physical ATM. This limits an organization’s ability to utilize offshore resources, limits collaboration and requires a test lab filled with every ATM model a fleet owner wishes to test.
The variety of ATM functionality further compromises efficiency when organizations use traditional desktop ATM testing tools. For some segments of society, the appetite for cash can be effectively served by the barebones “cash and dash” ATM models often found at convenience stores. On the other end of the spectrum are the customers accustomed to the highly sophisticated, full-service machines (offering bill payment and other features).
Technology Surpassing Traditional Testing
Innovations like “cardless cash” are rapidly taking hold in the US. Given the introduction of Apple Pay and other mobile phone-based digital wallets, it logically follows that many consumers will expect the ability to similarly access cash using their phone as opposed to the traditional plastic card. This carries implications for both ATM software and hardware and extends to other logistical issues such as the ability of the consumer to unlock the unattended doors behind which many ATMs sit.
These factors may be why ATM testing is a more specialized field than much of traditional IT testing, requiring knowledge of a wide array of touch points and interdependencies, both internal and external. This limits the number of resources available to help an organization protect its brand and reputation at the ATM. That is why many organizations with legacy test programs too often use the 80/20 rule. This approach, which is not limited to ATM testing, focuses on the 20 percent of use case scenarios most likely to expose 80 percent of the flaws.
Even if this “rule” is applied accurately, it leaves the organization counting on it with a significant level of exposure. In most cases, however, our experience suggests that ATM network operators, along with other players in the financial services industry, generate far less use case coverage than they realize, often well below 20 percent. No one who is head of an ATM testing organization could sleep well knowing this.
And after all, it only takes one wayward scenario to cause a fault, which could cause a handful of transactions to be lost or create a network-wide outage. This means that just because a network has not been down for years, does not guarantee that it won’t be down tomorrow. Even if the person whose career rests on providing the appropriate level of testing considers this a good bet, successful gamblers always take steps to improve their odds. Thus, any organization can benefit from testing as many scenarios as possible, minimizing the window for unnoticed defects. Virtualization provides the opportunity to drive that number down much, much closer to zero than most organizations realize is possible.
Virtualization Improves Overall Testing
Virtualized testing solutions provide a repository for test cases and related media that is accessible, as required, by the members of a testing organization. The test case repository grows over time, as do its benefits to the organization. The need for users to recreate test scenarios decreases and the level of accuracy of new test scenarios increases, because users can use existing tests as templates to more accurately design new ones. In addition to these cataloged scenarios, virtual testing platforms store input from users across multiple installations providing an organization with the benefit of lessons learned by a broader community of testers using the solution.
Concerning issues of compliance, it is a common misconception that decreasing regulatory oversight reduces mandates and alleviates the strain on testing teams. Testing is required to determine if a coming change impacts the hardware or software elements within the system which the mandate will impact. Therefore, the rollback of a regulation will likely lead to the need for testing too since it, most likely, will create an operating change of some kind. Another “Catch-22” relates to the call by many organizations to decrease regulatory oversight so that IT resources can be freed to explore more areas where innovation is needed. More innovation, while good, will require more testing and better testing methodology to keep pace with technological and consumer demands.
