Selecting an open source project to adopt, reference or to participate in can be a difficult decision. Prospective users of OpenTAP or any other project should ask themselves
Is the project active (or moribund)?
How large and complex is the code base?
Will the code integrate easily into existing infrastructure?
Which licenses apply to the code base?
How secure is the project code?
Do project developers observe best practices and community standards?
This blog will examine each of the above factors for OpenTAP and highlight tools to accomplish the task.
There exist myriad tools for characterizing and assessing the code in an open source source project. Your most logical starting point is the toolbox supplied by the project repository; in the case of OpenTAP, we can find a range of very useful tools in GitHub (and previously, in GitLab). Beyond a project's hosting platform are several types of third-party tools, both free and fee-based, that scan for licensing, vulnerabilities, dependencies and other attributes.
Web-based free tools, e.g., Synopsys OpenHub
Open Source composition analysis tools like ScanCode and FOSSology
Free versions of commercial software composition analysis tools, e.g., Fossa
Commercial tools from Synopsys Black Duck, Fossa, Fossid (now part of Snyk), WhiteSource et al.
In additional to acquisition/licensing cost, these tools vary in ease of use, ability to work directly on cloud-based repositories (vs. premises/downloads), whether the tools themselves are cloud- premises hosted, comprehensiveness and depth of scanning, e.g., for dependencies and snippets.
In preparing this blog, the author used tools on GitHub and from OpenHub and Fossa.
The OpenTAP Project
GitHub itself lets project owners present a high-level description, usually as a README.md markdown file as well as other summary information. GitLab does the same.
Like GitHub, OpenHub not only displays README files (when present) for reference but also constructs a very help "Nutshell" description that calls out key statistics in bullet form. One of the more interesting statistics is an estimation of total development hours, calculated using Boehm's Cost Constructive Model (CoCoMo). According to this estimate, developers have spent 22 years collectively building and maintaining OpenTAP.
Gauging Project Activity
There are numerous ways to gauge the level of activity for a given project, and also tools to let you compare how active one project is vs. another. The easiest metrics come from the project's top-level directory, which in GitHub will show the dates for the most recent commits to the project and to project subdirectories. For a more organized view, GitHub offers the Insights tool that summarizes project activity with Pulse, a Contributors dashboard that plots commits over time and stats for the leading developers on the project, and a number of other options for adherence Community Standards, Code Frequency, Dependencies, etc.
GitHub's Insights displays contributions over time and highlights individual developers for their contributions.
OpenHub also offers a view of the number of active contributors over time along with bar charts of developer contributions by name (when available):
One additional nice feature of OpenHub is its project activity icon set, an easily digested visual display of the activity level and health of a project. OpenTAP receives a "High Activity" rating from OpenHub.
GitHub itself offers powerful code search and characterization capabilities as part of its general Search and Languages sections. OpenHub offers a more attractive set of charts that compare the programming language and license composition of the project.
All tools in use indicated that OpenTAP is 98% written in C# and mostly licensed under the MIT license, but it's quite interesting to explore the "long tail" of other languages and licenses.
The declared top-level OpenTAP license is MIT but individual files in the project are licensed under MIT, MS-NET, Apache and also variously dual and triple licensed files under BSD, MIT and Apache
In addition to code contained directly in a project, software exhibits dependencies upon other projects, code bases, libraries and so on - according to GitHub, OpenTAP has dependencies upon over 1,000 other pieces of code, some in the project's manifest files, others referenced directly in OpenTAP and even others referenced indirectly via known dependencies. The majority of OpenTAP's dependencies lie in Microsoft .NET.
As part of Insights, GitHub provides a Dependency Graph:
Given today's global threatscape, project repositories and code analysis tools all offer perspectives on the relative security of open source software projects. GitHub security tools provide an overview of existing Security Policy and highlight any Security Advisories to date, with statistics on known vulnerabilities. GitHub also provides APIs for integrating security checks into project build cycles.
OpenHub offers visitors a sliding scale for "confidence" and "vulnerability exposure" and awards OpenTAP the highest ratings for both. OpenHub also has the ability to track vulnerabilities across versions and time periods - there have been no detected vulnerabilities in OpenTAP for its entire history.
You can gain a deeper view of vulnerabilities present and other security issues for a project by using (mostly commercial) SCA tools.
Like many other development and testing tools and frameworks, OpenTAP usually "lives in the lab" and is not user or internet-facing, such that even if OpenTAP itself or its dependencies do exhibit vulnerabilities, in most circumstances they would present minimal or zero risk of exploitation. However, novel emerging use cases exist (e.g., for home automation); moreover, OpenTAP is finding deployment scenarios in complex equipment that embeds testing functionality (especially in communications), such that securing the project is increasingly a real-world concern.
GitHub provides a unique metric - how a project compares to "community standards", that is, how complete is the set of artifacts that accompany the project. In particular, it checks for the presence of a project description, README file, contribution policy, code of conduct, key templates, and perhaps most importantly, a LICENSE file.
This blog has provide a short tour of both the state of the OpenTAP project and tools you can use to gauge the status and health of OpenTAP or any other open source project. In future blogs, we'll delve further into areas covered above, especially dependencies and security.