Open source software (OSS) simultaneously enjoys a superb reputation as a source for cybersecurity tooling and less-than-stellar (if undeserved) repute for suffering from vulnerabilities that facilitate data breaches and other attacks. Given that OpenTAP is an active open source project, supported by an array of open source plugins, members of the OpenTAP ecosystem often puzzle over the security of this test automation platform and of OSS in general.
This blog is not intended to be exhaustive. Instead, it explores the most relevant and pertinant issues in OSS security and how they apply to OpenTAP and OpenTAP plugins.
Before addressing whether OpenTAP is secure, let’s review the practices that help secure open source code bases.
Transparency vs. Obscurity
The security of open source software begins with a counterintuitive presumption, that the more open and visible software is, the most secure software. Here, Eric Raymond’s oft quoted Linus’s Law applies, stating that “given enough eyeballs, all bugs are shallow”. The same logic, in general, applied to detecting vulnerabilities.
The presumption is that more review is possible with open source, whereas such purview is impossible with closed / proprietary software, but not always practical. There are not always a sufficient number of expert eyeballs available, as was evident for infamous vulnerabilities like HeartBleed and ShellShock, and most recently, Log4j. Perhaps a more truthful mantra would be “given sufficient available expert reviewers, most vulnerabilities can enjoy remediation”.
Conversely, security-by-obscurity suffers from a false sense of serenity. Windows and other fully closed/proprietary code bases endure an even greater number of reported vulnerabilities (and surely an even greater number undiscovered and unreported). Remember – no source code was required to construct most Windows viruses, worms and rootkits.
The OSS Triple Fence
The term “triple fence” has been used to refer to physical defenses at secure government sites and was more recently employed to describe the ill-conceived fences along the U.S. southern border. Open Source Software, for its part, boasts a triple fence, providing impressive but not impenetrable security to the OSS supply chain.
The three primary fences are
Open Source Projects – project maintainers and their peers do not accept patches and new code sight-unseen. Some zealous maintainers refactor and re-write all contributions to meet their coding standards and notions of elegance. Increasingly, OSS projects, especially those under foundation umbrellas, automate the ingress of new code and employ various types of scanning tools to ensure the integrity of inbound code.
Open Source Distributions and Platform Creators – commercial distribution vendors (Red Hat, SUSE, Canonical, Oracle et al.) and creators of embedded toolkits and SDKs (OSVs, semiconductor vendors and others) look upstream to stable projects and also perform their own original development. Given their corporate and government customer bases, these distributors are obliged by best practices, by government edict (e.g., White House EO 14028) and in some cases by law to scan and vet inbound code and to provide software bills of materials.
End-users and other Deployers – while the average individual end-user is seldom equipped to inspect, scan or otherwise vet inbound OSS, enterprise end-users, application developers (ISVs) and integrators are better positioned to inspect and remediate OSS security issues in the code they manage, integrate and deploy.
The fences, separately and together, succeed in keeping out faulty and malicious code (or at least minimize its impact) and to let in genuine contributions and bug fixes
A fourth line of defense lies in projects and initiatives designed specifically to improve the security of open source in general by establishing best practices and remediating issues with critical technology (e.g., the lack of maintenance of the bash shell, SSL etc. as seen in 2014-2017). These projects include the ToDo Group, the OpenSSF initiative (which subsumed the earlier Core Infrastructure Initiative) and several others.
Vulnerabilities in Open Source
It is staggering to consider the total number of vulnerabilities reported each year in the National Vulnerability Database (NVD - maintained by MITRE and NIST). In the first half of 2022 alone, 11,242 were reported in NVD, on track to exceed the 20,168 discovered and reported in 2021. The total number of CVEs (Common Vulnerabilities and Exposures) continues to rise even as remediation resources do not.
A most pertinent question is how many of these reported vulnerabilities lie in open source code vs. proprietary. In past years, open source accounted for between 40-45% of reported vulnerabilities (IBM). You may think that such a value grossly underrepresents the real threat, since open source is so ubiquitous – one library can be linked by hundreds of different applications. Conversely, you may feel that the remaining 60% of vulnerabilities reported in proprietary code egregiously understates the true number in closed source, since security researchers may easily examine open source code but not proprietary software, leaving more vulnerabilities undiscovered.
Starting in 2015, suppliers of Software Composition Analysis (SCA) tools that target open source (Black Duck/Synopsys, Snyk, Veracode, White Source, et al.), began offering vulnerability detection and tracking to tool sets that previously focused solely on license scanning. These tools do not themselves find vulnerabilities in source code, but rather identify versions of OSS libraries and other components in a software build and check those against CVEs tracked by NVD.
OpenTAP and OSS Security
Dependencies
OpenTAP is an open source project with dependencies on multiple other projects, and so is subject to the vagaries of vulnerabilities in OSS code. The most visible and important dependency is with Microsoft .NET framework.
Microsoft .NET framework, of vital importance to OpenTAP and OpenTAP plugins, has a mixed history of total reported and resolved vulnerabilities. Learn more at .NET Open Source Security Insights.
In the lifetime of the platform, there have been 59 CVEs logged in the NIST National Vulnerability Database (NVD).
However, .NET also has a high percentage known vulnerabilities with high severity ratings, and rather complex dependency chains to a very broad API set.
OpenTAP Security Fences
Like the rest of OSS, OpenTAP enjoys the oversight of overlapping communities:
· Dedicated maintainers, at Keysight, Nokia and other OpenTAP ecosystem participants
· End-users, integrators and other contributors
· Product development teams at Keysight, who build hundreds of PathWave products on the OpenTAP platform
OpenTAP Use Cases
OpenTAP is first and foremost a test automation platform, deployed in the lab and on the test bench, across the end-product lifecycle. In this role, OpenTAP is thereby inward-looking: while not air-gapped from corporate and public networks, the vast majority of use cases are not customer- or Internet-facing.
Exceptions to isolated deployment include use in distributed and remote test environment, embedding in large infrastructure equipment to implement self-test, and in novel non-test applications, e.g., home automation.
Given its nominal exposure to outside threat, OpenTAP is unlikely to rise to the top of your security concerns. But don’t ignore it – your software and hardware under test could still be subject to abuse and test data to exfiltration.
Conclusion
Open Source Software, while the “gold standard” for security tools, has earned a mixed reputation for overall security and exploitability. That being said, the transparent core security practices around open source development and distribution, favor detection and remediation by a mix of community and commercial ecosystem interests. OpenTAP, as an open source project, benefits from this transparency and the potential for greater purview, and provides a very trustworthy foundation for test automation.
This blog just scratches the surface of open source security. Check this space in the future for more blogs and discussion on OSS security in general and the security of OpenTAP test automation.