Posted on May 17, 2024 by Stefan Prodan

CNCF projects highlighted in this post

Flux logo Helm logo Kubernetes logo

Project post originally published on Flux’ blog by Stefan Prodan

We are thrilled to announce the release of Flux v2.3.0! In this post, we will highlight some of the new features and improvements included in this release.

Helm banner of Flux v2.3.0 General Availability Release of helm-controller and the Helm Kubernetes CRDs

General availability of Flux Helm features and APIs

This release marks a significant milestone for the Flux project, after almost four years of development, the helm-controller and the Helm related APIs have reached general availability.

The following Kubernetes CRDs have been promoted to GA:

The Helm features and APIs have been battle-tested by the community in production and are now considered stable. Future changes to the Helm APIs will be made in a backwards compatible manner, and we will continue to support and maintain them for the foreseeable future.

Enhanced Helm OCI support

The HelmRelease v2 API comes with a new field .spec.chartRef that adds support for referencing OCIRepository and HelmChart objects in a HelmRelease. When using .spec.chartRef instead of .spec.chart, the controller allows the reuse of a Helm chart version across multiple HelmRelease resources.

Starting with this version, the recommended way of referencing Helm charts stored in container registries is through OCIRepository.

Using OCIRepository objects instead of HelmRepository improves the controller’s performance and simplifies the debugging process. The OCIRepository provides more flexibility in managing Helm charts, as it allows targeting a Helm chart version by tag, semver or OCI digest pinning. If a chart version gets overwritten in the container registry, the controller will detect the change in the upstream OCI digest and reconcile the HelmRelease resources accordingly.

Example of a HelmRelease referencing an OCIRepository:

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: metrics-server
spec:
  interval: 10m
  chartRef:
    kind: OCIRepository
    name: metrics-server
  driftDetection:
    mode: enabled
  values:
    apiService:
      create: true
---
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: OCIRepository
metadata:
  name: metrics-server
spec:
  interval: 12h
  layerSelector:
    mediaType: "application/vnd.cncf.helm.chart.content.v1.tar+gzip"
    operation: copy
  url: oci://docker.io/bitnamicharts/metrics-server
  ref:
    semver: ">=7.0.0"

Improved observability of Helm releases

By popular demand, the helm-controller now emits Kubernetes events annotated with the Helm chart appVersion in addition to the version info. When configuring alerts for Helm releases, the appVersion is now available as a field in the alert metadata and is displayed in the notification messages. The appVersion field is also included in the HelmRelease status, and in the gotk_resource_info Prometheus metrics.

When using an OCIRepository as the HelmRelease chart source, the controller will also include the OCI digest of the Helm chart artifact in the Kubernetes events and the HelmRelease status.

Benchmark results

To measure the real world impact of the helm-controller GA, we have set up benchmarks that measure Mean Time To Production (MTTP). The MTTP benchmark measures the time it takes for Flux to deploy application changes into production. Below are the results of the benchmark that ran on a GitHub hosted runner (Ubuntu, 16 cores):

ObjectsTypeFlux componentDurationMax Memory
100HelmChartsource-controller25s40Mi
100HelmReleasehelm-controller28s190Mi
500HelmChartsource-controller45s68Mi
500HelmReleasehelm-controller2m45s250Mi
1000HelmChartsource-controller1m30s110Mi
1000HelmReleasehelm-controller8m1s490Mi

Compared to Flux v2.2, in this version the memory consumption of the helm-controller has improved a lot, especially when the cluster has hundreds of CRDs registered. In Flux v2.2, helm-controller on Kubernetes v1.28 runs out of memory with only 100 CRDs registered. Whereas, in Flux v2.3 on Kubernetes v1.29, it can handle 500+ CRDs without issues. Given these results, it is recommended to upgrade the Kubernetes control plane to v1.29 and Flux to v2.3.

Image update automation improvements

The ImageUpdateAutomation API has been promoted to v1beta2 and the image-automation-controller has been refactored to enhance the reconciliation process.

The v1beta2 API comes with a new template model that can be used to customize the commit message when the controller updates the image references in the Git repository. The commit template supports old and new values for the changes made to the files containing the policy markers. In addition, the commit message is included in the Kubernetes events emitted by the controller, offering better visibility into the automation process.

The ImageUpdateAutomation API now supports selecting ImagePolicies using label selectors in the new field .spec.policySelector.

Migration to v1beta2 template model

To migrate to the v1beta2 API, update the apiVersion field in the ImageUpdateAutomation resources to image.toolkit.fluxcd.io/v1beta2, and modify the messageTemplate to use the Changed template data.

Example template:

apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImageUpdateAutomation
metadata:
  name: <automation-name>
spec:
  git:
    commit:
      messageTemplate: |-
        Automated image update
                
        Changes:
        {{ range .Changed.Changes -}}
        - {{ .OldValue }} -> {{ .NewValue }}
        {{ end -}}
        
        Files:
        {{ range $filename, $_ := .Changed.FileChanges -}}
        - {{ $filename }}
        {{ end -}}

Example generated commit message:

Automated image update

Changes:
- docker.io/nginx:1.25.4 -> docker.io/nginx:1.25.5
- docker.io/org/app:1.0.0 -> docker.io/org/app:1.0.1

Files:
- apps/my-app/deployment.yaml

For more examples and details, see the ImageUpdateAutomation documentation.

Signatures verification with Notation

The Flux source-controller now supports verifying the authenticity of OCI artifacts signed with Notation (CNCF Notary project).

To enable Notation signature verification, please see the following documentation:

In addition, the Flux CLI now supports generating Kubernetes secrets with Notation trust policies, using the flux create secret notation command.

Big thanks to Microsoft for contributing to the development of this feature!

Terraform provider improvements

The Flux Terraform provider has undergone a major refactoring and now supports air-gapped bootstrap, drift detection and correction for Flux components, and the ability to upgrade and restore the Flux controllers in-cluster. Starting with this release, the provider is fully compatible with OpenTofu.

The provider documentation has been updated with examples and detailed usage instructions.

New maintainer

We are very happy to announce that Steven Wade has joined the Flux project as a maintainer of the Terraform provider. Steven has been a long-time contributor to the Flux project and we are excited to have him on board!

Controllers improvements

CLI improvements

Breaking changes and deprecations

Deprecated fields have been removed from the HelmRelease v2 API:

The following APIs have been deprecated and will be removed in a future release:

Supported versions

Flux v2.0 has reached end-of-life and is no longer supported.

Flux v2.3 supports the following Kubernetes versions:

DistributionVersions
Kubernetes1.28, 1.29, 1.30
OpenShift4.15

Flux v2.3 is the first release end-to-end tested on OpenShift. Big thanks to Replicated for sponsoring the Flux project with on-demand OpenShift clusters. For more information on how to bootstrap Flux on OpenShift, see the OpenShift installation guide.

Enterprise support

Note that the CNCF Flux project offers support only for the latest three minor versions of Kubernetes.

Backwards compatibility with older versions of Kubernetes and OpenShift is offered by vendors such as ControlPlane that provide enterprise support for Flux.

Installing or upgrading Flux

To install Flux, take a look at our installation and get started guides.

To upgrade Flux from v2.x to v2.3.0, either rerun flux bootstrap or use the Flux GitHub Action.

To upgrade the APIs in the manifests stored in Git:

  1. Before upgrading, ensure that the HelmRelease v2beta2 YAML manifests are not using deprecated fields. Search for valuesFile and replace it with valuesFiles, replace patchesJson6902 and patchesStrategicMerge with patches.
  2. Commit and push the changes to the Git repository, then wait for Flux to reconcile the changes.
  3. Upgrade the controllers and CRDs on the cluster using Flux v2.3 release.
  4. Update the apiVersion field of the HelmRelease resources to helm.toolkit.fluxcd.io/v2.
  5. Update the apiVersion field of the HelmRepository resources to source.toolkit.fluxcd.io/v1.
  6. Update the apiVersion field of the ImageUpdateAutomation resources to image.toolkit.fluxcd.io/v1beta2.
  7. Commit and push the changes to the Git repository.

Bumping the APIs version in manifests can be done gradually. It is advised to not delay this procedure as the deprecated versions will be removed after 6 months.

What’s next for Flux?

The next milestone for the Flux project is v2.4, which is planned for Q3 2024 and will focus on the image automation APIs and S3-compatible storage APIs. For more details on the upcoming features and improvements, see the Flux project roadmap.

After the introduction of OCI Artifacts in 2022, we had a recurring ask from users about improving the UX of running Flux fully decoupled from Git. In response, we made a proposal for a flux bootstrap oci command and a new Terraform/OpenTofu provider that relies on container registries as the unified data storage for the desired state of Kubernetes clusters. The RFC can be found at fluxcd/flux2#4749 and we welcome feedback from the community.

Over and out

If you have any questions, or simply just like what you read and want to get involved, here are a few good ways to reach us: