A frequently debated topic is whether open source software (OSS), presumably free to acquire, ends up costing more or less than proprietary software over its lifespan. This blog series examines the factors that influence Total Cost of Ownership (TCO) for OSS and how they differ from proprietary software, developed and sourced commercially, in-house, and from open source, with particular attention to TCO for test automation.

“TCO, or actual cost, quantifies the cost of the purchase across the product's entire lifecycle. Therefore, it offers a more accurate basis for determining the value -- cost vs. return on investment (ROI) –than the purchase price alone. – Steven Bigelow, TechTarget”

The Scope of Costs

The Total Cost of Ownership (TCO) for an asset is defined as the Acquisition cost plus the cost of Operation. For software, Acquisition can involve licensing, subscription, and/or development by an organization or under contract to third parties.

Operation is more expansive and can include costs for a proof-of-concept, integration/ operationalization, training, provisioning, compute time (e. g., on a public cloud), support, upgrades, maintenance, security-related expense and even costs associated with end-of-life.

Other expenses that can impact TCO include license and regulatory compliance, and quality assurance.

“TCO = Acquisition + Operation”

TCO for Test Automation

Discussing TCO across the breadth of IT would surely occupy volumes. The focus of this more modest blog series is constrained to TCO in Test Automation.

Test automation costs differ from enterprise or desktop software costs:

Software acquisition costs are often embedded in the cost of developing one or more products / devices under test (DUTs), or in the cost of test equipment

Stand-alone commercial / shrink-wrapped software is a rarity

But like bespoke enterprise software, test automation tools and platforms

Are developed and maintained in-house
Don’t follow commercial update/upgrade schedules
Usually lack formal documentation and training materials

Costs in Context

The total set of costs associated with an IT asset do not accrue in a vacuum. Rather, they originate with the lifecycle of that asset and the resources associated with it.

IT Asset Lifecycle

TCO analyses tend to focus on acquisition cost, but total costs span the entire lifecycle of an asset, from planning to end-of-life (and sometimes beyond).

Typical lifecycle costs include

Planning/Management and Staffing

Development/Acquisition Costs
Integration
Implementation / Workloads
New Project Launch or Legacy Migration
Data Center or Cloud Tenancy
Maintenance and downtime
Compliance
End-of-Life

Let’s examine how these lifecycle costs apply to test automation

Planning and Staffing

For test automation, staff are usually already in place in roles that include test engineering, QA and QC. Increasingly, with adoption of agile methods and “shifting left,” staff involved in decisions about test automation will also include core developers.

For open source software, best practices suggest that organizations assign a “code owner” to manage the lifecycle of a software component, including interaction with the associated developer. Code ownership seldom demands an entire full-time equivalent (FTE), but the demands of the role should not be underestimated. In some organizations, code ownership is shared between primary stakeholders (developers, testers, et al.) and staff in an Open Source Program Office (OSPO).

With in-house software, a full development project must also be planned, including staffing, timelines, equipment, tools and support. In cases where in-house systems are developed in an ad hoc or piecemeal manner, little up-front planning is typically performed; however, experience shows that the same (or greater) management time will be expended over the life of the project to deal with these issues.

Next Time

In the next blog in this series, we'll dive into perhaps the most complex component of Total Cost, Acquisition.

Total Cost of Ownership for Test Automation