Project post cross-posted from the Weaveworks blog by Tamao Nakahara, Head of Developer Experience, Weaveworks and Flux community manager
Challenge
The DoD knew that it needed GitOps. Nicolas M. Chaillan, Chief Software Officer of the U.S. Air Force states that “The U.S. Department of Defense is the largest organization on the planet with over 100,000 developers. At that scale, you have to manage against node drift. We knew that GitOps was the solution to manage drift and to enable automation. The CNCF Flux project was a clear choice for us at Platform One because it provided fully-baked support for Helm, which provides a robust ecosystem of tools for our end users.” Read below for their use of Flux for U.S. Air Force U-2 jets and more!
Solution
Chaillan and team created Platform One, a DevSecOps platform for their many teams to build software safely. They wanted to use “CNCF-compliant Kubernetes clusters and other open source technologies across the DoD.” From the beginnings of Platform One, the team at the DoD designed it for declarative repeatability, automation, centralized configuration management, as well as Kubernetes cluster management, workloads, and zero trust architectures using service meshes.
| For Nicolas M. Chaillan, Chief Software Officer of the U.S. Air Force, “Configuration as code was a shining star. The CNCF Flux project was a clear choice for us at Platform One because it provided fully-baked support for Helm, which provides a robust ecosystem of tools for our end users.” |
For Chaillan, “Configuration as code was a shining star.” Once they set up the GitOps tooling, they could set up improved automation.
The DoD were early adopters of many CNCF technologies and Chaillan worked to make the DoD the first government organization to become part of the CNCF. In addition to being powered by Flux and Helm, Project One leverages other CNCF projects including Jaeger, Open Policy Agent (OPA) Gatekeeper, Fluentd/Fluentbit, Kubernetes, Prometheus, Argo, and Envoy (Istio).
Impact
Flux and Helm are the only two projects in the Adopt category by the CNCF Tech Radar for CD! So it’s no surprise that the combination of the two projects would provide a fantastic solution.
For DoD contractors on the projects, Tom Runyon (Defense Unicorns) and Josh Wolf (Rancher Federal), Helm provides access to a whole ecosystem of community and corporate backed tools. Because Flux uses the native Helm SDK (using Flux’s Source Controller and Helm Controller), the team can take advantage of the vast Helm ecosystem and experience available for their GitOps needs.. The result is a richer experience for their end users within various departments in the DoD (for instance, the Air Force, Navy, etc.). As Wolf shares, “for the end users, all of your Helm knowledge and tooling transfers directly when using Flux because releases are still deployed as Helm releases using all the Helm hooks, etc.”
| Contractors Tom Runyon and Josh Wolf trust Flux to deploy and manage their software on Air Force U-2 jets. Wolf says enthusiastically, “the Flux API is crazy stable!” |
Runyon and Wolf trust Flux to deploy and manage their software throughout their work with Platform One, Air Force, and Navy. Runyon is leveraging Flux and the Platform One ecosystem to create streamlined developer workflows for the U.S. Navy, and deploying apps on Navy boats. In addition, Wolf’s use of Flux on top of the CNCF certified and sandbox distribution k3s combine for a lightweight, automated, and declarative tactical edge deployment on military systems such as the Air Force’s U-2 DragonLady. A huge benefit for them is that Flux provides enterprise-level reliability. As Wolf says enthusiastically, “the Flux API is crazy stable!” Runyon adds, “With Flux, we can make upgrades seamless and regular. We can roll out weekly updates to Party Bus (the SaaS version of Platform One). Flux’s great backwards compatibility also means that we don’t need tight coupling between versions of Flux with versions of other software. Unlike with other solutions that have required tighter coupling (and more work for us), Flux gives us the freedom not to put additional cycles to that challenge.” The team uses Flux to roll out almost everything: Helm releases, Argo, Jaeger, and almost all of their tools.
The team uses Flux to deploy Helm charts, and the charts are used for all app lifecycle management. In addition, as Wolf notes, “Big Bang (Platform One’s CD tool that uses Flux) uses both Helm and operators. We use Helm to install those too.”
Finally, monitoring is a critical part of the process and Flux’s integrations with Prometheus and Grafana are essential for that.
For Runyon and Wolf, game-changing Flux capabilities include:
- Flux’s Notification Controller: For Runyon “this is great for managing systems, making sure updates get rolled out, and getting alerts when things fail. It provides a great user experience.”
- dependsOn: “This is my favorite Flux capability!” says Wolf because “it provides fully declarative directed async dependencies. You tell it what you want to depend on and it will figure out the chain of dependencies for you. With other solutions, you have to choose and manage the dependencies manually.”
- SOPS support built-in: Obviously, for the government, security and secrets are critical. Since Flux is opinionated with SOPs (unlike other solutions that ask that you “bring your own solution”), Wolf shares that the SOPs support helps them to centralize on a solution easily.
- valuesFrom: For the government that prioritizes security for sensitive information, Flux makes it possible to have all values loaded from secrets instead of being committed to git.
- postRenderers: This simplifies the team’s customer-facing support for bug fixes in an otherwise fairly complex workflow that includes maintaining custom work and upstream contributions. It does so by, instead, providing a simple patching mechanism. As Wolf notes, “Helm’s postRenderer can be a little tricky so Flux’s API makes it easier. The CRD for postRenderer is very clean.”
How you can Get Started with Flux and Helm!
- Follow our Guide for using Flux with Helm: https://fluxcd.io/docs/use-cases/helm/
- Set up monitoring with Prometheus and Grafana: https://fluxcd.io/docs/guides/monitoring/#install-flux-grafana-dashboards
- Wolf and Runyon also recommend looking at the k8s@home community for great tips (for example this Template for deploying k3s backed by Flux).
Top things to know about Flux:
- Flux is a tool for keeping Kubernetes clusters in sync with sources of configuration (like Git repositories), and automating updates to configuration when there is new code to deploy.
- Flux is built from the ground up to use Kubernetes’ API extension system, and to integrate with Prometheus and other core components of the Kubernetes ecosystem. In version 2, Flux supports multi-tenancy and support for syncing an arbitrary number of Git repositories, among other long-requested features.
- Flux powers the GitOps offerings of Microsoft, AWS, VMware, D2iQ, Weaveworks, and other companies. Learn more on October 20.
- Flux provides GitOps for both apps and infrastructure: Flux and Flagger deploy apps with canaries, feature flags, and A/B rollouts. Flux can also manage any Kubernetes resource. Infrastructure and workload dependency management is built in.
- Just push to Git and Flux does the rest: Flux enables application deployment (CD) and (with the help of Flagger) progressive delivery (PD) through automatic reconciliation. Flux can even push back to Git for you with automated container image updates to Git (image scanning and patching).
- Flux works with your existing tools: Flux works with your Git providers (GitHub, GitLab, Bitbucket, can even use s3-compatible buckets as a source), all major container registries, and all CI workflow providers
- Flux works with any Kubernetes and all common Kubernetes tooling: Kustomize, Helm, RBAC, and policy-driven validation (OPA, Kyverno, admission controllers) so it simply falls into place.
- Flux does Multi-Tenancy (and “Multi-everything”): Flux uses true Kubernetes RBAC via impersonation and supports multiple Git repositories. Multi-cluster infrastructure and apps work out of the box with Cluster API: Flux can use one Kubernetes cluster to manage apps in either the same or other clusters, spin up additional clusters themselves, and manage clusters including lifecycle and fleets.
- Flux alerts and notifies: Flux provides health assessments, alerting to external systems, and external events handling. Just “git push”, and get notified on Slack and other chat systems.
- Flux has a lovely community that is very easy to work with!
Author: Tamao Nakahara, Head of Developer Experience, Weaveworks and Flux community manager. See www.gitopsdays.com for the next Flux ecosystem event.