Total Cost of Ownership for Test Automation - Part III

This third blog in our series examines the factors that influence Total Cost of Ownership (TCO) for OSS and how those costs differ from proprietary software, developed and sourced commercially, in-house, and from open source, with particular attention to TCO for test automation.

Migration from Legacy

At some point in the history of an organization, a software solution or a software category will be acquired or developed for the first time. But equally often, organizations find themselves with the need to replace a legacy solution or system, including requirements to support new instrumentation, with accompany requirements to develop new drivers, interfaces, APIs, listeners, etc.

Moving from an incumbent solution to a new software tool or platform does not happen spontaneously. It is a process with associated costs.

As with any type of legacy software, moving to a new test automation platform presents risks in configuration, programming, and operation, and requirements for that move:

Need to learn a new testing framework, a different architecture, or an unfamiliar programming language, and retarget legacy code, scripts, test cases, etc. to it
Ensure compatibility and interoperability with existing test software and hardware instrumentation
Establish stability and quality of the migration target
Verify utility and usability of solution outputs
Avoid interruption or degradation of existing operations, e.g., ability to collect results data and maintain testing continuity

Proof of Concept

Rather than taking a “leap of faith”, a proof-of-concept lets you test and evaluate important metrics and parameters in a real-world setting. Costs for a POC can encompass acquiring software for testing, reconfiguring legacy hardware or acquiring new prototypes, training, physical space and possible cloud tenancy.

Learn more about OpenTAP PoC.

Training

Training costs seldom account for a major share of expenditures but are not insignificant, especially for large teams. Training expense can include tuition (for standardized training), instructor fees (for in-house training), training development (for customized or 100% custom training), travel costs for off-site education, materials and repro-graphic costs for syllabi, study guides and manuals, meals and other incidentals.

For open source, and for broadly used proprietary software (think Windows, an Office Suite or popular databased), a big advantage is that training is usually available through multiple channels and over a range of media -in-person, remote, web-based, etc. Vertical and specialty applications, and bespoke software, open or closed, will have few COTS options and the greatest need for custom-built training. As with any type of software, training for open source is usually a worthwhile investment.

Legacy Retention

Just because you are migrating to a new platform does not mean that you can say goodbye to the incumbent software. If at all possible, organizations should carry forward and reuse key elements of legacy test automation, into the PoC, as a practical measure and to reduce costs.

However much or little of the incumbent test infrastructure is carried forward, legacy test code, test platform(s), documentation and infrastructure should be captured and archived. In many cases, the replacement with new technology will be incremental, raising costs by requiring support for two (or more) solutions.

Integration and Operationalization

Once you determine the actual viability and cost for a test automation solution, you still need to “plug it in” to existing workflows, or modify those work flows. Those workflows can include system, platform and application builds, module and system testing, QA/QC and beyond.

Approaches

This integration typically follows one of two approaches: “wrapping” the new interfaces to match legacy patterns or modifying existing workflows to accommodate new tools, frameworks, file formats, etc.