This second blog in our series examines the factors that influence Total Cost of Ownership (TCO) for OSS and how those costs differ from proprietary software, developed and sourced commercially, in-house, and from open source, with particular attention to TCO for test automation.

Acquisition Costs

Make vs. Buy?

Make vs. Buy is an eternal question facing IT and engineering management in any company. Open source changes the nature of this question, in that both buying and building will almost definitely involve some amount of open source software.

Acquiring software can take various forms:

Licensing shrink-wrap commercial software
Developing software in-house
Downloading open source software
Hybrids of the above three

The Role of Open Source Software

In today’s marketplace, it’s next to impossible to buy commercial-off-the-shelf (COTS) software that does not integrate at least some open source. Even the most proprietary commercial applications, tools or platforms will likely include dependencies on open source libraries, middleware and other subsystems. However, the open source items in COTS software will have minimal incremental impact on buying and managing that software (unless you plan to resell or transfer it later).

In a similar fashion, when building even highly bespoke software, smart developers turn to open source components and tools. Surveys reveal that today’s enterprise and device software stacks integrate large portfolios of open source – 90% of IT leaders are using open source today [Red Hat], for infrastructure modernization, application development and digital transformation. Given the efficiencies offered by leveraging open source, it is seldom (or never) worth the effort of banishing OSS from your software portfolio and reimplementing from scratch.

“90% of IT leaders are using open source today – Red Hat”

Finally, you may choose to acquire and deploy community OSS, either downloaded as ready-to-run binaries or built from project source trees. This option has the lowest initial acquisition cost but likely carries incremental cost for integration and operationalization with other parts of your software portfolio and workflows.

Acquisition Cost Factors

Different types of software present diverse commercial models to the acquirer. Test automation is no exception.

Commercial Acquisition – Perpetual License

Organizations have been acquiring commercial/proprietary software for decades with a perpetual license, one that grants usage rights to a site, across an entire organization, or to some number of users or computers for an unlimited period. An associated parameter is the cost of support, which is usually contracted on an annual basis at additional cost, for access to technical assistance, bug fixes, product updates, etc.

Subscription Model

More recently, many software vendors have migrated from perpetual licensing to subscriptions for both premises software and SaaS, billed monthly or annually. For desktop or workstation software, this cost scales by the number of users enabled. For server software, the cost scales by the number of server blades, clusters or cores. For embedded software, customers (OEMs) are usually charged a mix of project fees, development “seat” fees and run-time license charges. With test automation, subscription cost scales based on purchased test hardware, instances of test platforms and tools installed on workstations, and/or number of supported users

Downloaded Open Source Software

For open source downloaded directly from a project repository, the acquisition cost is effectively zero (limited to actual time spent by staff on downloading and installing). Alternately, acquiring organizations can choose to receive project code from a vendor, typically a company active in developing and maintaining the open source project of interest. Suppliers of commercial open source often also employ a subscription model, with costs comparable to proprietary software, but usually less.

Developing Software In-house

While open source and COTS software can provide a high-value catalog of software, organizations may still want or need to develop portions of their software stack in-house. At a minimum, such software may constitute “glue” to support integration and operationalization of third-party code. Conversely, the scope of in-house custom coding may comprise a much larger swath or even the majority of the deployment stack, with an accompanying jump in costs.

“We do these things not because they are easy, but because we thought they were going to be easy – Jeff Atwood”

Generally speaking, end-to-end in-house software development is a legacy practice today. Building code from scratch is expensive, time-consuming and risky. Risks include resource availability, resident expertise, meeting (and disrupting) schedules, quality and supportability. However “easy” in-house coding may appear, it is seldom planned with precision and is too often the source of delivery shortfalls and cost overruns. These surprises are even more likely when in-house development is undertaken in an ad-hoc or piecemeal fashion.

A comparison between the real costs of in-house platforms vs. OSS and commercial proprietary test automation software is complicated by the incremental nature of most in-house development. Legacy test platforms evolve from simpler test scripts and drivers but arrive at the point of comparison after generations of modification. Moreover, each generation is usually unaccompanied by formal methods or metrics. Given this historical blank slate, we can only compare the available state and cost of in-house software in its final form, as a whole, based upon the number of lines of code (KLoC) in such platforms. While such an in toto analysis may seem to inflate the cost, it actually underestimates it because it cannot capture the number, overhead, and other circumstances of prior releases.

Acquisition Costs in Test and Measurement

Most or all of the acquisition models listed above exist for test and measurement software. Vendors of traditional legacy tools continue to offer perpetual licensing, with upfront acquisition costs plus annual charges for support. Other T&M vendors have converted to subscription models in concert with other software types and segments.

OpenTAP, as an open source test automation platform, is free of charge to download, install and run. Instrument-specific interfaces and other plugins may also be available as open source, without acquisition cost, or may be commercialized by hardware vendors in the OpenTAP ecosystem. Other OpenTAP-compatible software, such as tools, results viewers, etc., are also available under open source and commercial models.

Keysight’s commercial version of OpenTAP, PathWave Test Automation is available both under perpetual licensing and via subscription. Keysight also embeds OpenTAP into complete hardware solutions, accompanied by support charges.