Posted on May 14, 2021

Last December, as we were all counting down the days until 2020 was no more, FireEye posted an announcement that caused various beveragewares across the world to fall and shatter on the floor: some sort of state-sponsored attacker had stolen their red-teaming tools. Why would a state-sponsored attacker— especially one who had already clearly demonstrated they could break through a security firm like FireEye’s defenses (reminding us all that even security professionals are fallible)— need the tools FireEye themselves used to break through other’s defenses? Within a week, it became clear that this was just the tip of the iceberg. 

These attackers, whose identities are yet to be confirmed but whose initials probably rhyme with SVR, had compromised SolarWinds Orion to deliver trojanized updates to clients all across the globe. As became clear in the weeks that followed, this attack, along with similar exploits of Microsoft and VMWare products, had exposed and compromised thousands of organizations, including NATO, the UK Government, the EU Parliament, the US Departments of Commerce, Defense, Homeland Security, Justice, State, Treasury (and more), Microsoft itself, and numerous other state and local governments and private sector entities. The true expanse of this attack is yet to be determined.  The compromises went back at least 9 months, possibly longer, and represented a stunning breach of many high-profile targets through a supply chain vector — a documented growing area of concern by organizations including the Linux Foundation and the CNCF.

Preventing supply chain attacks is still a nascent, rapidly developing field. While the CNCF Security Technical Advisory Group (TAG) has documented instances of supply chain compromises going back at least to 2003, with the cadence of such attacks picking up substantially since about 2017, there is still little general consensus on how to prevent such attacks. This is why the Security TAG has published a new paper, Software Supply Chain Security Best Practices, designed to provide the cloud native and open source communities with a holistic approach to architecting a secure supply chain regardless of whether they are a software producer or consumer.

The approach outlined by the CNCF Security TAG group has four key elements:

First, establishing and verifying “trust” at every step in the process through a combination of code-signing, metadata, and cryptographic validation. Essentially, the goal is to establish a “we trust because we’ve verified” architecture for each point within the entire supply chain.

Second, “automation”: everything that can be automated should be automated and documented. This helps to prevent accidental errors and makes it easier to spot when things have gone awry.  

The third element is “clarity.” This takes two forms. Every step in the software build and supply chain process should be clearly defined with a precise, limited scope. Next, deriving from this design, every actor within the supply chain (whether they be human or machine) must have a clearly defined role. This, in turn, allows us to limit the permissions of these actors to exactly those needed.

Finally, every entity in our system must engage in “mutual authentication.” This means that no human, software process, or machine should be trusted to be who they say they are, they must demonstrate through a hardened technique (such as MFA) that they are in fact who they purport to be. The mutuality of this (developer demonstrates their identity to server, server simultaneously demonstrates their identity to developer) helps to prevent exploits such as Man-in-the-Middle or replay attacks.

These principles can be operationalized across first party source code repositories, third party dependencies, build pipelines, artefact repositories, and deployments. Each of these stages involves different considerations and requirements, and comes with its own set of best practices. As a companion, we are also providing an “adoption framework” in this blog post that can be used to assess your own supply chain architecture and help identify the areas where you may need to place the most emphasis.

This paper fits within the broader work of the CNCF Security TAG group. Last year, the group published a paper on Cloud Native Security, emphasizing the importance of automation, shifting security left, and putting security issues on the same footing as other software “bugs” or development priorities. The CNCF Security TAG is also working on:

Within that body of work, this paper continues the emphasis on automation and treating security as a first order concern in the software development process. We outline many of the tools available for supply chain security. Indeed, one of the threads running through this paper is that while developing a secure “software factory” is a significant undertaking, most of the pieces to do so are readily available for teams willing to take up the challenge. Our hope is that by elevating the priority of security within software producing organizations, we can establish a clearer baseline that enhances the security of us all. This is especially true of supply chain security concerns, which, as we have seen, can have a wide ranging impact not just on the breached organization itself but on their downstream and indirect customers, too.

A Framework for Supply Chain Evaluation

So how does your supply chain stack up? Here are some questions to ask yourself:

Verify Source Code

Verify Materials

Protecting Build Pipelines

Protecting Artefacts and Deployments