Conducting your SAP test automation orchestra

E2E testing is always needed to validate the business process flows in SAP environments, and traverses both SAP and other connected systems. As the organization’s ERP platform, SAP typically sits at the heart of the business, with multiple dependencies between different SAP modules and non-SAP systems that are tightly integrated, as we all know. It’s a complex landscape. And there is usually more than one test automation tool in use across this landscape, which adds to the complexity.

Business assurance

There’s no silver bullet in terms of an E2E test automation strategy needed to assure the business that the process flows are doing what they’re supposed to do. Nonetheless, we see a couple of different approaches being adopted with a good measure of success.
The first approach is to go all-in on a single test automation platform across all applications and systems. Most modern commercial automation platforms support a wide technology stack. However, there are some challenges with this approach. Through experience, we know that every automation tool is ideal for some technology stacks, but not very consistent or stable for other technology stacks. So, it might be fantastic for SAP, but when you step outside the platform you start noticing cracks appearing with the automation, or vice versa. For example, a tool that works phenomenally well with the SAP GUI but fails to recognize some dynamic drop-down objects in the CRM SaaS solution will present problems with automation of the entire E2E process flow. This is a typical hurdle for the effective use of test automation in end-to-end testing.

A new conversation

An alternative strategy is to change the conversation from test ‘automation’ to test ‘orchestration’, giving you the ability to automate using different tools / frameworks across the E2E landscape. Here, you choose the right testing tool for each technology stack or application within the process and then orchestrate it so that it is married together to form the end-to-end testing flow.

Take the order-to-cash process as an example This is a fairly standard E2E test in most SAP implementations, extending from creating a sales order and promising a delivery date, through packaging, shipping, delivering and ultimately invoicing. If the entire process is implemented on SAP, it isn’t all that challenging to automate the E2E test. But what if you need to automate parts of an E2E test that is something beyond the standard? For example, validating that the generated invoices at the end of the process are in the right format, for the correct amount, and whether credit has been applied, etc. These checks are often done manually, which can be painful and time consuming. For a global implementation with multiple inter and intra company order-to-cash flows, this could mean several hours of invoice validation. But there are tools for automating them to speed up the process.

One simple solution we have used for this specific example is the Apache PDFBox open source library, which is built in Java. This library can be embedded into any Java-based automation framework for automated comparison of PDFs or images. The comparison is achieved with high accuracy and efficiency. Since this comparison is done programmatically, bulk comparison of PDFs and images can be carried out very quickly. But now we end up with two separate tools and frameworks to validate different parts of the E2E process. Can that actually be an automation strategy? Yes, it can.

Validating change

Another specific use case from a client engagement is a legal entity name change. This had to be reflected across all forms for all impacted company codes. The number of forms to be validated (factoring in multiple languages) ran into the hundreds. That’s a lot of regression testing! We used PDFBox to introduce an automated PDF comparison process that validated each document against a ‘golden’ PDF copy. We orchestrated the automated order-to-cash SAP transactions in one tool followed by the PDF validation steps in our Java framework using Microsoft Azure DevOps. All within a touchless end-to-end test automation flow.

Orchestrating tools

Another use case I want to reference is that for a client in the oil and gas sector. The company had an SAP ECC system integrated with a standard manufacturing operating management (MOM) solution for its refineries. It had a standard commercial test automation product in use, which didn’t work on the MOM solution. This presented problems with automating the end-to-end tests. We did some preliminary checks to find out that the robotic process automation tool (RPA) tool the client had licenses for was able to identify the UI objects in the MOM solution. In this case, we orchestrated the testing within the automation tool itself and used an RPA (bot) to pick up the test flow from the automation tool. The bot performed some steps in the MOM platform and handed it back to the test automation tool. Our orchestration of the different tools ensured the MOM component was tested and then handed back to SAP as part of the end-to-end process in a closed loop. Once again, the orchestration created a touchless test automation flow.

Adopting touchless test automation

A strategy for end-to-end touchless test automation in SAP environments and solutions should look at how best to get value for money. Mixing and matching automation tools to manage both your SAP and non-SAP processes gives you choice. But it is only with a clear orchestration strategy that the true value can be realized.

Once your tools are brought together in a harmonized approach, you can start to reap the benefit of touchless automation. For example, setting up your automated testing to run in a schedule of non-peak hours makes life easier in testing complex end-to-end processes continuously.