Automated Testing for Cloud-based ERP Applications

Published On : March 1, 2017

Yogi Berra famously said, “You can see a lot by just looking.”

I could paraphrase him in the context of testing of cloud-based ERP applications and say, “You can test a lot by just testing.”

Let me explain …

One great benefit of the new breed of cloud-based applications is their flexibility. This allows systems to be agile and to respond quickly to new business processes. However, this means that changes are always being made to the systems environment by way of customizations and integrations. The other set of changes originate in the release upgrades initiated by the cloud software vendor.

As adoption of cloud ERP increases, customers have an urgent need to validate their processes for regression after each change. Manual approaches are tedious, while automation frameworks require coding. So how do you “test a lot by just testing”?

Well, clearly you need to start with an automated testing solution. However, apart from automation, the solution needs a number of other characteristics for it to be effective. I have listed a few of these below.

ERP Application-specific

In their ‘2015 Magic Quadrant for Software Test Automation’ report, Gartner made the following comment in reference to vendors of generic software test automation products:

“In general, we find that everyone “supports” testing of packaged application software — in that if it is Windows or Web, it is “just another application.” However, this ignores that packaged application testing scenarios are different to most custom made software.”

So the automated testing tool for cloud-based ERP applications should be specifically designed for these applications.


Automation frameworks require scripting or coding and this immediately eliminates the possibility that the Business or QA Analyst can define the test cases themselves. The tool should be designed with the analyst in mind so that he/she can define and execute the test cases with a little training.

Easy Test Data Capture

The automated testing tool should provide a simple mechanism to capture data to be used to create test cases. The test data may represent transactions, entity records, or customized forms. It should provide a way to control navigation from one test case to the next in a process flow such as “Order to Cash”. In addition, it should allow editing and copying of test cases.

Assemble Process Flows

Business processes implemented in cloud-based ERP systems are highly integrated, so testing an individual transaction is of limited value. This means that it should be possible to assemble a process flow from individual test cases. Also, since regression testing needs to be performed frequently, it is necessary to allow any setup needed for test data to make each test execution self-contained.

Incorporate Integration Steps

Many business processes involve integrating data from/to a non-ERP system. In order to automate testing of a process flow, the tool should allow triggering of integration as a part of the flow.

Integrate Issue Management

There is a need to log an issue or a case whenever an error is reported during testing. The test automation tool should provide an ability to integrate with popular issue management systems. This has the benefit of automating the issue reporting process.

User and Role-based Execution

An essential element of testing is to confirm that access to various functions is restricted by user and role. The testing tool should be able to simulate of the same. In addition, it should be possible to inspect the results of a test case by viewing screenshots and the generated test transactions.

Success Criteria

The analyst should be able to easily define criteria that assess if the test case has completed successfully. This assessment should allow a comparison of the actual values to expected values in the base record or its associated records. This enables the tool to scrutinize the impact of any customizations that are expected to trigger upon saving of the original record.


There is a need to save test groups and process flows and edit them to reflect new users, roles and success criteria. This collection of test cases should then be available as a repository for future test cycles.


Finally, the test automation tool needs to provide an easy-to-use reporting capability. Queries and reports need to summarize data by key attributes such as test cycle, transaction types and user/role. This enables efficient management of the testing effort.

To conclude, I will quote John Ruskin who said,

“Quality is never an accident. It is always the result of intelligent effort.”

Some tools which meet these criteria have been introduced in the market. (e.g. Xcelero for NetSuite) and managers of cloud-based ERP applications need to leverage these to provide the intelligent effort necessary to deliver quality service to their customers and minimize the risk of regression.