Calculating the Return on Investment (ROI) of Test Automation

Many of our existing customers and some potential new customers come to Fusion Alliance wanting to know more about test automation and if it’s right for their business. The answer to that question, the following assessments must be done to determine the argument for or against using test automation.

1. Assess the maturity level of the customer’s software testing department and their processes.
2. Assess the nature of the engagement Fusion Alliance is being asked to assist the customer with.

These assessments are essential as many organizations make the mistake of thinking that test automation will immediately reduce the costs of testing, increase test coverage, and shorten test cycles. These are sometimes the longer-term benefits of test automation, but are not immediately evident. To determine if test automation will indeed reduce costs for the organization in the long term, the ROI must be calculated. The ROI for test automation is simply the benefit of test automation divided by the cost of test automation.

The cost of test automation is higher than manual testing and should include any costs related to hardware, software and licenses, time for resources to produce scripts, and cost of the resources themselves.

The benefits of test automation must be calculated over a period of time for the software under test and take into account the reduced time to execute tests and the ability to test as frequently as the organization wishes. These figures should be compared to the costs of manually testing the same software.

Think of the cost of test automation vs. the cost of manual testing as follows:

Cost of test automation = price of hardware needed + price of tool required + time to develop test scripts + (time to maintain test scripts x number of times test scripts are executed) + (time to execute test scripts x number of times)

Cost of manual testing = time to develop test scripts/charters + (time to maintain test scripts/charters x number of times tests are executed) + (time to execute test scripts/charters x number of times)

After figuring the costs of test automation and manual testing, calculate ROI as follows:

ROI = (cost of manual testing – cost of test automation) / cost of test automation

By now, you might be saying to yourself “that seems too easy”. If you are saying this, you would be correct. Other factors must be taken into account such as the benefits of manual testing over test automation. The most complex parts of the software should be left to a manual tester to analyze to ensure maximum quality. Therefore, a mix of test automation and manual testing might be the best solution. Context must also be taken into consideration. What is the complexity of the software? What is the size of the software and what is its role in the business? Smaller, less complex applications that won’t take a considerable amount of time to test won’t require test automation because the upfront costs will outweigh the benefits gained. Also keep in mind the types of testing you are looking to automate. Each of these different types will have a different ROI that needs to be taken into consideration.

Thinking about all of these things upfront will not only help you determine whether to automate your testing or not, but also help you plan your overall testing effort and give you a jump start on your test plan(s).