Open source spirit: elevating team collaboration and innovation

Ambassador post by Leo Pahlke, CNCF Ambassador and CNCF TAG Environmental Sustainability Chair

Open source is a fascinating space, where you are surrounded by emerging technologies and where you can directly engage with and have an impact on their future. “Engineering on the shoulders of giants.” All of this in a collaborative, open environment where you learn and contribute at the same time.

I’ve been thinking more about the space lately, which led to writing this article. In times when we think about sustainability, not just in terms of the environment and climate, but also in terms of social and economic sustainability, how we like to work together, how we can drive further innovation and build new technologies, open source is an interesting contrast. I hope and believe that we can learn from it and adapt practices not only between open source communities, but also within companies.

This article explores concepts of how open source software is done based on my experience. What are the driving forces? Why are innovative projects often open source projects? How are companies adopting open source in their business? What might be the next steps for the field? Are there any lessons we can learn from open source? There are plenty of questions that point to different matters. Hopefully, there are bits and pieces that help you understand the space better and give you fuel to build on your companies open source strategy. Okay, let’s talk about my latest open source snapshot.

1 – Tinkering

Open source began as a movement decades ago and has continued to evolve. It involves collaborating on software projects and other matters, such as data or standards. In this article, we will focus on open source software. Open source approaches vary depending on cultural contexts, perspectives, preferences, and your goals and interests. Open source software is driven by its contributors and maintainers and not companies. Engineers who spend time freely pushing a project forward and sharing their work with others. Engineers not only set the entire picture, adopters (companies, institutions) and open source foundations are also part of it, but it all stands on engineers building software and sharing their work.

One of the core reasons open source is possible lies in the nature of software—it can be shared with little to no cost, as software is a form of information. This may sound strange and irrelevant, but this opens up a lot. Software teams can spend years developing a product, but once it’s finished, it can be copied and shared across networks almost instantly.

Engineers generally like to tinker, experiment, and learn new technologies, and, like everyone else, show and share with others our work. Unlike some other engineering fields, software engineering makes tinkering and experimenting easy. No costly manufacturing, supply chains, or expensive equipment are required, meaning you don’t need company or university backing. This leads to some interesting effects, one of which is the potential to democratize software innovation. With some basic computing power and curiosity about software, you are set up to journey through tech. In this regard, the barriers to shaping the digital world are low.

Together, these two factors explain why open source emerged and became a widespread movement. Beyond those, there are other practical reasons why openness and collaboration can benefit your company goals, as explored a bit later.

Wonder and curiosity sparks exploration and experimentation, which can lead to the creation of new projects. If we share our work, this leads eventually to an ocean of projects, with a few that stand out and gain further traction. This creates a self-reinforcing cycle, where successful projects attract more attention and contributions. Since these projects are open, engineers can learn and experiment and build on top of the project as a base. Over time, this creates a network of interconnected projects, where each one influences the others. Influential projects can build over time an ecosystem that is defined by APIs, culture, and mindsets.

If you have experience collaborating in open source communities, and put it in contrast to a company you worked for, processes are vastly different. Both generate value. In the open source space, it is possible to experience that, if conditions are a bit different, the entire motion of how software is developed can become a different twist. Open source is chaotic, and maintainers put simple guidelines to make it work. Which ideas can we take from open source and bring to our engineering teams at our company? Think about it.

And of course, this is slightly more complex. In the context of open source projects, social structures emerge, encompassing communities that collaborate, organize meetings, establish contribution platforms, establish communication channels, and so forth. It is not only about innovation and building new technologies, but also about community, learning, and other things. »Linux Kernel people have their approach to building open source than the React or the Hyperledger community«.

With time, I think there are multiple reasons that emerged why folks end up contributing to open source. Tinkering and experimentation being at the core of it. Purpose, change, fellowship in communities, reputation, experience, freedom to build, and passion are some more.

From my experience, it’s usually a mix of all of these topics that people get into open source because of. It’s not always a rational choice, but rather a path that presents itself, and you take a look.

2 – Movement in IT

With new open source projects coming up, the IT landscape shifts over time. The IT landscape has been influenced by open source for decades. It’s nothing new. Companies like Google have been founded with open source in their DNA, embracing free and open services and releasing new open source projects. Others, like Red Hat or SUSE, are building their business model on top of it, expanding on open source software to offer enterprise versions and support services to companies. Others, such as Microsoft, have further advanced their company strategy by partially abandoning proprietary software in favour of a hybrid approach that encompasses open source. Everyone mentioned uses open source software, but they have their ideas, plans, and priorities.

Adopting Open Source

The companies’ engagement in open source is not triggered by wonder and the freedom to create, as in the previous chapter defined for engineers, but rather simply by creating business value. Can open source be used to generate more business value. Yes, it can!

If your company is concerned with software, it becomes clear that software engineering is a complex endeavour. To manage complexity, we capsule logic and divide and conquer. That’s important to deal with complexity but also opens up the possibility to share your projects with others effectively. Components can come from open source or be developed internally; ultimately, it doesn’t really matter: you use interfaces. The result matters. Results generate business value. Open source helps you to achieve your goals faster, to focus on your business value. Therefore, open source is about efficiency. It’s also about reducing engineering complexity you need to deal with. Since the project is capsuled, all the business logic (should be) safe and secure with you, and you do not lose your competitive edge.

However, there is a crucial prerequisite for open source to be effective: you must separate your project’s core business logic from the more general components that can be shared and reused. Open source thrives when it focuses on extracting general ideas, approaches, and algorithms that are applicable across various contexts. From my experience, open source is about creating generic solutions that serve as foundational building blocks. In a way, it shares similarities with fundamental research: just as research seeks to uncover universal truths about the world, open source seeks to define and share fundamental technologies that can be built upon and adapted in many settings.

There are more reasons for companies to engage with open source, as shown in the diagram and table after.

At the same time, there are several risks to consider when adopting open source as a strategic goal. The nature and impact of these risks will vary on a case-by-case basis. One of the primary risks is the commitment to taking ownership of your software stack. Contributing to and maintaining open source projects requires investments in engineering excellence—not just in writing code, but in ensuring long-term sustainability, quality, and security. Additionally, software engineering is in some ways a creative process that can be difficult to manage, and this may require a shift in mindset. It’s not just about the tools; it’s about fostering a culture where engineers take ownership of the entire project, not just their individual contributions. This shift places added pressure on talent acquisition and retention, as companies must attract engineers with the skills and mindset to thrive in such an environment. Finally, the total cost of ownership with open-source solutions can be less transparent than with proprietary alternatives. While open source can reduce licensing fees, the costs associated with maintaining, integrating, and securing these tools can accumulate. However, if a company aims to be a leader in the digital space, these risks are likely ones that will need to be addressed anyway.

Open Source Commitment: A Nuanced View

Open source plays a role for every company that does software engineering; however, the engagement is vastly different. There is quite a discrepancy between using open source because software engineering is nowadays not possible otherwise and being an open source citizen. To invest in open source requires driving factors, which we explored in the previous section. These could be transparency-related, for example, or expanding the monetization model. This driver needs to be picked up and integrated into the business strategy. If that’s the case, more active engagement and leadership can be achieved.

Investing in open source and becoming an active open source citizen may still be a goal for the future, but not something to realize soon. Organizations are still in the midst of their digital transformation, working to reach a level of maturity that would enable them to play a more active role in the open source community. Open source itself exists on a spectrum, ranging from exclusive reliance on proprietary software to fully embracing the open source spirit. A balanced middle ground may be a practical approach during the digital transformation. Engagement with open source can be gradual, allowing companies to build capabilities and expertise over time while transitioning towards more active involvement.

Strategies for companies to adapt open source varies. To figure out how to manage your open-source engagement within the company, you can take a look at the resources by the TODO Group.

3 — Empowering Contribution

Alexis de Tocqueville suggested that change arises when there is pressure from both the top and the bottom. This can be interpreted in several ways: there needs to be a clear vision, guidance, and direction from the top, while commitment, energy, and passion from the bottom drive progress. A vision alone won’t create change, and simply putting in the work doesn’t guarantee it either. This dynamic can be understood in both large and small contexts. For example, on one hand, maintainers at the “top” create and release software; on the other, companies, and developers at the “bottom” provide the energy and motivation to adopt, modify, and advance it and their products or service. Change happens. Guiding and empowering this process is key to facilitating meaningful change and encouraging ongoing contribution.

Open source foundations play a vital role in building communities, setting guidelines, and fostering collaboration. They sit in the middle as a “neutral body” since, open source thrives on collaboration rather than confrontation. This is where open source foundations come in—they help negotiate between the interests of maintainers and adopters and empower contribution. While some open source projects thrive without the backing of a formal foundation, a neutral body is often essential for nurturing a strong community and ensuring the long-term success of both the project and its technology stack. Without such an entity, it can be much more difficult to build and sustain a community, which in turn hampers the future development of the project.

One example of how this is done can be observed by looking at the structure of the CNCF, where you have different structures bringing expertise and representing the interests of maintainers and end users. Together, we decide on governance and collaboration guidelines.

4 — Elevate Collaboration and Innovation

My journey through open source projects and communities so far showed me an interesting contrast to regular software engineering practices. Open source complements companies software engineering capabilities every day. Playing a more active role in open source comes with some risks due to the chaotic nature of the space. Still, I believe that to be a leader in technology, it’s important to take these risks intentionally and benefit from shaping technology not just for your customers but also for everyone else. As explored, there are plenty of motivations for it.

A question remains when more traditional companies that rely on software now and in the future to stay competitive will reach digital maturity to contribute to open source like digital natives. For some companies, they reached this stage. In this light, it’s to be seen how the space evolves going forward. It is difficult to create a collaborative and innovative environment in a company. Open source can inspire to imagine a more collaborative and democratic process in software engineering going forward.