A frequently debated topic is whether open source software (OSS), presumably free to acquire, ends up costing more or less than proprietary software over its lifespan. This blog series examines the factors that influence Total Cost of Ownership (TCO) for OSS and how they differ from proprietary software, developed and sourced commercially, in-house, and from open source, with particular attention to TCO for test automation.
The Scope of Costs
The Total Cost of Ownership (TCO) for an asset is defined as the Acquisition cost plus the cost of Operation. For software, Acquisition can involve licensing, subscription, and/or development by an organization or under contract to third parties.
Operation is more expansive and can include costs for a proof-of-concept, integration/ operationalization, training, provisioning, compute time (e. g., on a public cloud), support, upgrades, maintenance, security-related expense and even costs associated with end-of-life.
Other expenses that can impact TCO include license and regulatory compliance, and quality assurance.
TCO for Test Automation
Discussing TCO across the breadth of IT would surely occupy volumes. The focus of this more modest blog series is constrained to TCO in Test Automation.
Test automation costs differ from enterprise or desktop software costs:
Software acquisition costs are often embedded in the cost of developing one or more products / devices under test (DUTs), or in the cost of test equipment
Stand-alone commercial / shrink-wrapped software is a rarity
But like bespoke enterprise software, test automation tools and platforms
Are developed and maintained in-house
Don’t follow commercial update/upgrade schedules
Usually lack formal documentation and training materials
Costs in Context
The total set of costs associated with an IT asset do not accrue in a vacuum. Rather, they originate with the lifecycle of that asset and the resources associated with it.
IT Asset Lifecycle
TCO analyses tend to focus on acquisition cost, but total costs span the entire lifecycle of an asset, from planning to end-of-life (and sometimes beyond).
Typical lifecycle costs include
Planning/Management and Staffing
Implementation / Workloads
New Project Launch or Legacy Migration
Data Center or Cloud Tenancy
Maintenance and downtime
Let’s examine how these lifecycle costs apply to test automation
Planning and Staffing
For test automation, staff are usually already in place in roles that include test engineering, QA and QC. Increasingly, with adoption of agile methods and “shifting left,” staff involved in decisions about test automation will also include core developers.
For open source software, best practices suggest that organizations assign a “code owner” to manage the lifecycle of a software component, including interaction with the associated developer. Code ownership seldom demands an entire full-time equivalent (FTE), but the demands of the role should not be underestimated. In some organizations, code ownership is shared between primary stakeholders (developers, testers, et al.) and staff in an Open Source Program Office (OSPO).
With in-house software, a full development project must also be planned, including staffing, timelines, equipment, tools and support. In cases where in-house systems are developed in an ad hoc or piecemeal manner, little up-front planning is typically performed; however, experience shows that the same (or greater) management time will be expended over the life of the project to deal with these issues.
In the next blog in this series, we'll dive into perhaps the most complex component of Total Cost, Acquisition.