TAG post originally published on Cloud Native sustainability’s blog by Joe Kutner
Container builds can be very wasteful. Every operating system update, new dependency version, and toolchain upgrade results in copious amounts of energy used to build and rebuild our container images; often unnecessarily. It can be costly at scale, which is why Cloud Native Buildpacks were designed to perform full builds only when a rebuild is actually required.
Buildpacks transform application source code into container images. They can be used with or without Docker to encapsulate common patterns across builds, which makes containerization easier and more consistent for app developers. Buildpacks also provide advanced caching and patching mechanisms that make them an environmentally friendly choice for container builds. In certain cases, Buildpacks prevent many images from being rebuilt at all. That’s a big shift from other cloud native technologies that may assume unlimited cloud resources are available.
On the Environmental Impact of Cloud Native
Prior to the emergence of the cloud native ecosystem and widespread use of container images, we deployed our applications in servers built from machine images that were updated infrequently and on a different cadence from the application itself.
Today, many applications are coupled to the operating system and its packages because they use a Dockerfile to define their container images. As a result, those images frequently need rebuilds to apply patches to OS-level components, or simply to update tools that aren’t even used by the application. Even worse, the layer caching mechanism imposed by Dockerfile forces us to frequently rebuild layers that don’t even need to be rebuilt.
The cloud-native ecosystem has brought great productivity and operational improvements to software development. But we’ve lost sight of how wasteful some of those technologies can be. Buildpacks, on the other hand, have been designed to work at a scale (i.e. tens millions of images) where being wasteful has real costs. That’s why the Buildpacks rebase mechanism requires minimal resources.
Reduce, Reuse, Rebase
Container images built from a Dockerfile require a full build when a new operating system update is available, even if your app doesn’t need a re-compile or re-install to work with the update (i.e. the update is ABI compatible). This is not the case when using Buildpacks.
When a new operating system base image becomes available for an image that been generated by a buildpack, the existing layers that sit above the operating system can be reused. This process, illustrated below, is called rebasing. The application layers, with the exact same SHA, can be lifted on to the new operating system image layers.
Buildpacks rebase process ultimately constructs a new container image using both the existing layers and the new operating system layers, without the need for build. At its core, image rebasing is a simple process. By inspecting an app image, rebase can determine whether or not a newer version of the app’s base image exists (either locally or in a registry). If a newer version exists, rebase updates the app image’s layer metadata to reference the newer base image version. This is essentially a an operation that edits a JSON file. It takes milliseconds and uses very little compute resources.
Rebase allows app developers or operators to rapidly update an app image when its run image has changed. By using image layer rebasing, this command avoids the need to fully rebuild the app.
You can learn more about rebase in the Buildpacks documentation. But rebase isn’t the only Buildpacks mechanism that’s more sustainable than Dockerfile builds. Buildpacks can also cache build artifacts to enable incremental compiliation, and other resource saving techniques. These cache layers won’t always be discarded when you do require a re-build, like they would with `Dockerfile“ builds.
Be As Green As Your Unit Tests
Container builds are not the biggest offender when it comes to the environmental impact of software. Electricity needed to mine bitcoin is more than used by entire countries, but the growth of software that uses cryptographic techniques has brought new awareness to how our code affects the world around us. That’s a good thing.
We have a responsibility to think about minimizing the required resources of the software we produce. The code we write has an impact on the world, and our choices matter.
To learn more about the relationship between open source software development and the environment, visit the Environmental Sustainability Technical Advisory Group (TAG)