The use of open source in networking for 2017: An overview
Open source is paradoxical: It's easy to understand what it is and why it is attractive to many different parties; what's much harder is to understand how it can be monetized in various business, product development and delivery models. In a set of four spotlight reports (of which this is the first), we will examine open source as it is used today in networking, where it plays an especially important role because of both the move to networking delivered in a software form factor and the transformation of networking to incorporate software-defined capabilities.
The 451 Take
The use of open source software and the participation in open source communities is becoming a necessary part of the networking business, but one that requires significant changes in business and organizational models if the greatest benefit is to be achieved. There are new lessons and behaviors to be learned, ideally without doing the wrong thing and learning from the mistake. 451 Research has created this set of reports to help in that process.
What is open source?
Open source is best understood as a very different way of building software, rather than a way of distributing or using software. Open source software is built by a voluntary and open community of developers driven by the collective interests in the community, whereas historically most software products are defined, funded, built and sold by a vendor. The software created by an open source project is available for use without any license fee, and can be used as part of a commercial product. For a combination of reasons that will become clearer below, the legal and governance structures of open source projects and use are important, so most significant projects are done under the aegis of one of a few nonprofit organizations (e.g., Linux Foundation) with well-defined, battle-tested contracts that define participation, governance and use.
The question of whether software should be free and shared was a vibrant debate through the early days of home and personal computing, and was ongoing when Bill Gates and Paul Allen created Microsoft's first BASIC offering. At first, home hobbyists made copies of BASIC and shared them freely within the community. Bill Gates responded by arguing publicly and forcefully within that community that software had real value and had to be paid for, creating the foundations for the PC software industry and the basis for his wealth.
Computers are now abundant and of little value without software. Since the 1960s, the cost of computing has come down by a factor of roughly 1,000,000 (Moore's Law of twofold improvement every three years for 50 years), so the economics are inverted: Hardware is essentially free; the value is in the software. Despite that, open access to software is still meaningful, but for quite different reasons. Some of the key ones are presented below.
The poster boy for the value of open source is Linux, an open source operating system modeled after Unix that began more than 25 years ago. Linux was created by Linus Torvalds (who also led the community). Unix, originally created by Bell Labs, had evolved into multiple proprietary variants. The Linux community created an open version that supported the well-defined Unix functions, and then proceeded to carve out (make free) the high-value elements of the various proprietary Unix version. The result (over time) was the erosion of the proprietary Unix market and the support of Linux by the vendors in that ecology. Red Hat, the most successful open source company, was founded on a supported version of Linux (Red Hat Enterprise Linux).
Why is open source attractive?
There are many reasons, which we will discuss individually and consider in various combinations.
- Developer betterment and hiring benefits.
- A better development process.
- A way of industry competitors collaborating legally.
- A way of simplifying software product licenses and sales.
- A way of simplifying support and partnership by providing access to the source code.
- A way of building software more cost-effectively.
Developer betterment and hiring benefits
Many developers are highly motivated to participate in open source efforts because the project is more interesting than what they do at their 'day jobs,' and it is an opportunity to work collaboratively with a stronger team than they normally can. This developer motivation in turn influences vendor and customer hiring – many software vendors believe you can get more and better job applicants if working on open source projects is part of the opportunity. The same benefit applies to hiring networking programmers, especially as increasingly more open source code is involved in networking.
Hiring good programmers with the expectation that they can work on open source projects acts as an accelerant for the adoption of open source within the business. It requires employers to come to grips with key employees spending 'work time' on projects not under the direct control of the company, increases internal awareness of external open source activities, and increases the pressure to integrate open source into the business strategy and development plan.
A better development process
Open source can materially improve the talent that can be brought to bear on programming projects. The best programmers are a lot better than an average programmer. Most organizations have a mix – enough really good programmers to make good design and implementation decisions (if they can hire them, which can be challenging), and then many more average programmers to make coding faster and to do maintenance. Open source projects are a way to get more exceptional programmers involved, which can make a very material difference, subject to the constraint that you can't simply tell them what to do.
The value of multi-company collaboration is even higher in networking and communications because interoperability is often the essence of networking, and interoperability can be achieved more quickly with participation from multiple companies with different technical specialties and different markets and customers served. This benefit is particularly relevant in networking because deep networking expertise is rare (the details of creating networking software is not something you can easily learn in school).
Open source collaborative efforts are also proving to be an effective way to develop standards, especially in networking and communications, where interoperability is essential. Traditionally, network standardization starts with a written specification – for example, defining the inter-device coordination protocol and desired semantics. But important nuances often aren't understood until the protocol is coded for multiple different switch software implementations. Creating the standard as code, with ongoing integration and testing of that code in various implementations, has proven an effective way of collapsing the time required for this process, as well as strengthening the technical resources contributed to standardization efforts and getting better standards.
A way of industry competitors collaborating legally
As everything becomes more software-intensive, there is growing motivation for industry participants – competitors – to work with each other and share software. Such collaboration can enable both parties to move faster and focus their development efforts on the aspects of a product or marketplace where differentiation can provide commercial value. This is quite relevant to networking, given that interoperability is the essence of networking – everything has to ultimately work together.
Unfortunately, direct collaboration between competitors (in the case of software) is viewed with suspicion – with good reason, since it often led to price fixing. As a result, it is scrutinized and burdened by government regulation. Open source is an attractive alternative – collaborating with competitors via joint participation in open source projects. Open source has a mature legal framework that represents a reasonable and accepted solution to the competitive, commercial and regulatory issues that are intrinsic to industry collaboration on software development.
Again, there are many tradeoffs and new skills for an enterprise to learn as it becomes more involved with open source. Competitive politics don't go away just because you can collaborate with a competitor legally. Major contributors to open source projects that are competitors will try to drive the technical direction of the project to their advantage, with regard to what is added to the project and what is left as complementary (unshared software), and how the external interfaces to the software are defined.
Making effective use of open source participation ultimately depends on embracing very new strategies and new notions of products, including these new forms of competition. A real concern is that some collaborators may choose to hold back code that creates a competitive advantage rather than share a feature for the betterment of the community.
A way of simplifying software product licenses and sales
Small software vendors can significantly simplify selling to large companies if they use open source software. Large companies insist on contracts that make clear their continuing rights to the product in case the vendor becomes insolvent or is acquired. In the past, this has required the creation of an escrow account that stores the exact source code from which the customer's product is built (keeping it up to date as the product is revised and bugs are fixed), as well as a contract that defines the circumstances that trigger the customer's access to the code in escrow and their usage rights. This process is greatly simplified if the software is under the control of an open source project and the vendor commits to building its product from a known version at the cost of eliminating proprietary value. Many open source companies begin with a commercial offer of support for 'free' software (the successful Red Hat business model).
A way of simplifying support and partnership by providing access to the source code
Making some or all of a networking product's source code available to customers can have clear value when it comes to supporting the product, as has been demonstrated by Cumulus Networks. Cumulus says that the fact that its customers have access to the product source code often enables them to provide more detailed descriptions of problems, in turn accelerating Cumulus' ability to resolve and remediate them. The source code is a valuable form of precise documentation, even if production use is done with a specific binary distribution (a standard product). The use of a product binary is particularly relevant in networking, where the objective is a reliable and correctly operating network.
The essence of networking is interoperability. The standards specify the communications protocols used to coordinate the connected devices, but they don't specify the internal (software) implementation of the protocol, which will be different if the products come from different vendors or from different product lines from a single vendor. Reference to sources can be very useful when diagnosing this kind of interoperability issue.
The same kind of reasoning applies to technical partnerships that require some level of integration. Having much of the product open source enables a potential partner to understand in considerable detail the opportunities and challenges without requiring nondisclosure agreements or detailed technical briefings.
A way of building software more cost-effectively
As companies get more familiar with open source (the opportunities and challenges of participating in open source projects, the implications of how products can be monetized), they better understand how contributing proprietary code to an open source project might be in their selfish best interests (AT&T said as much about its decision to contribute ECOMP to open source). Having an open source community can accelerate development, speed problem resolution and repairs (to the benefit of the contributor), and accelerate the software lifecycle progress (leading to improved versions of the software being done faster).
For most software products or systems, the differentiated value comes from a small part of the software that reflects proprietary insight or intellectual property; conversely, most of the software contributes little to the propriety or differentiated value of the whole. If it makes sense to contribute that low-value part to open source (the issue is whether there is any community of interest in that part), then it can be a good idea – even from a selfish perspective. That is the hope of AT&T with regard to ECOMP. The tradeoff is giving others access to software that would cost a lot to develop for the benefit of community involvement with the ongoing maintenance and enhancement, and the potential value of the de facto standardization that it creates.
The nuanced view of a community
The idea of a development community is key to modern open source, and is more complicated and nuanced than one might expect at first glance. An open source project only works if there is, in fact, a community of interest, and that can't just be assumed to exist.
As a company becomes increasingly involved with open source, it will better understand where communities will and will not exist, and better anticipate where the community will come from and what the selfish interests of the other participants are. The constituents and motivations of the community have a profound impact on the outcome. Deciding to contribute something to open source is only part of the process.
Obligations of using open source software and the loophole
The simple idea of open source software is that it can be used and adapted for any purpose with a license fee, but that any modifications or improvements must be returned to the project for others to benefit from, as well. There is a very important loophole: The obligation comes if you sell a software product based on the open source, but not if you just use it or use it as part of a service you offer (such as Google GCP, AWS or Microsoft Azure). The commercial impact today is largely reflected by the use (without return) of open source by the hyperscale service providers.
Google, for example, started with Linux many years ago as a foundation for its search services. Google ended up building systems of unprecedented complexity and running them under very high load, and by doing so discovered and had to repair many elements of the Linux design, in essence creating an industrial-grade version of Linux that would have been valuable for many others, but because Google was building a service rather than selling a software product, the Linux license didn't require those improvements be returned to the community (and they weren't, despite the enormous potential community value in the Google fixes and enhancements).
There is a second nuanced implication in terms of what it means to support open source – use of open source in markets as a product attribute that helps the customer avoid vendor lock-in. Due to this loophole, although (for example) Google provides functions with the same interfaces as an open source offering, the Google implementation is very different (e.g., higher performance, more scalable). As a result, the customer may in fact be locked-in to Google because of superior performance, or at least not able to change ships without suffering consequences. Hence the use of open source standards is more complex than one might assume.
In today's world, where the hyperscale service providers have become major creators of new software technology (e.g., containers), Google is able to use open source creatively by releasing and contributing to important open source projects while building performance-optimized proprietary versions of the software internally. By doing so, it can very honestly advocate the value of open source software (prevents vendor lock-in) while using its engineering resources to create systems with very differentiated performance.