As open source software becomes mainstream and is integral to almost every software product, developers and their managers have an increasingly important role to play in assuring the quality and security of applications and physical products that contain software.

The Changing Approach to Open Source Code Adoption

In the past, developers often accepted open source software "as is", without reviewing it as thoroughly and carefully as they reviewed custom code built by their own team. This approach was particularly true when they adopted code from commercial entities or with foundation sponsorship. Even community-developed code would be accepted without detailed review, especially if it had many stars on its home page on GitHub. Fortunately, most of the code was satisfactory, so there were relatively few problems. The development organization had to write and test less code, enabling them to build and release their software more quickly.

Today, however, open source code has become more complex, especially with AI and machine-learning applications that apply heuristics based on language models and/or large bodies of data. It's often impossible to know how (or if) that code will work with different models or data. In addition, many open source components rely upon other open source components, which have been developed and released by different organizations, with significant dependencies and license issues. Beyond that, as open source software has become more widely trusted and used, it is being used in many critical applications, such as automotive or medical applications, where failures may be life-threatening or otherwise catastrophic.

As a result, organizations releasing software, with or without open source code, must assure that their software performs correctly, reliably, efficiently, and securely. While an Open Source Project Office (OSPO) may handle the responsibilities of licenses and business arrangements associated with any open source code, the responsibility for achieving these technical software qualities lies with their software development organization(s). Experienced software developers on the OSPO team can perform an initial review of the open source components, then rely upon the team using that code to carry out a more detailed analysis. The extent of needed analysis can be based upon the criticality of the final application, the maturity of the open source code, and its provenance.

Key Issues to Address in Open Source Code Reviews

Here are some of the key issues that should be addressed as part of this review:

1. Functionality - Does the open source component behave as expected, i.e., as described by its developers? Is it a good match for your organization's needs, or does it do too much or too little?
2. Code review - How well is the software architected? Does it fit well with the architecture of your code? Is the code well written and documented? Is it readable? Does the code rely on open source libraries that also need review? Do those libraries rely on yet other libraries (ad infinitum)? Can you create a Software Bill of Materials (SBoM) showing all of the code that is used in the open source component? Has that software been extensively tested on the same platform that you are using? Is there an available set of test cases that you can use for validation in your software product? If not, do you have to perform static and dynamic testing on the open source code as part of the review?

1. Code support - Who is responsible for supporting the code that you are reviewing? Is there a stable team of committers and/or a commercial development team?
2. Development process - Is there a team that is actively maintaining the open source code and issuing updates as needed? Is there a visible list of open and resolved issues? Is there a roadmap showing features, enhancements, and updates for planned future releases? (Some commercial developers treat these latter two as proprietary.) What will your organization do about updating those new updates in your so:ware?
3. Community support - If the open source component is not commercially released and supported, how strong is the community associated with it (beyond the team of committers)? Is the software mature, i.e., with numerous releases over time? How many people participate on message boards and answer questions posed by others? How active are those boards? Are they monitored and updated?

Code review is particularly important if you are planning to use an open source component for previously untested situations. While libraries will usually have a well-defined set of APIs and message formats, it may be necessary to create an interface between your application and the open source component. As noted, many open source components draw upon numerous libraries, which may involve a chain of other libraries. This chain may be very extensive, pulling in thousands of lines of code, affecting memory and storage use, as well as performance efficiency, which can make the component unsuitable for embedded applications and product.

Scalability is another potential issue, as some open source components may not have been evaluated under heavy user load; a component that works with 100 concurrent accesses may not work with 10000 or more, requiring you to perform stress testing before accepting a component. Then it is essential to validate the security of any component that will be put into production use, suggesting the importance of penetration testing and other dynamic testing, particularly if the software will be accessible over the Internet. While pen testing can be time-consuming and doesn't guarantee security, it can help to find common problems, such as open ports and SQL injection issues.

Finally, the development team coordinates with their Open Source Project Office by reporting the results of their evaluation. This report should cover the issues identified here, as well as any others that may be specific to the organization and its business(es). There may be some open or unresolved issues included in this report, leading to a decision to accept, reject, or further study the open source component.

Final Actions

The organization may have an individual or a small committee to decide how to proceed, recognizing that it is often OK to accept some risk in using a component, depending on the particular situation. Some organizations and people are more risk adverse than others, often based on the intended use of the software product and the consequences of failure. However, analyzing an open source component on the issues described here is an excellent way to identify the technical risks and decide how best to proceed.