Mastering Argo CD image updater with Helm: a complete configuration guide

Posted on November 5, 2024

CNCF projects highlighted in this post

Member post by Stanislava Racheva, DevOps & Cloud engineer at ITGix

Understanding Argo CD Image Updater

In modern Kubernetes environments, managing container images and ensuring that applications are always running the latest, most secure versions can be daunting. Argo CD Image Updater simplifies this process by automatically checking for new container image versions and updating your applications accordingly. Integrating seamlessly with Argo CD enables fully automated updates to Kubernetes workloads.

The beauty of Argo CD Image Updater lies in its simplicity and flexibility. By annotating your Argo CD application resources with a list of images and defining version constraints, the Image Updater takes over the heavy lifting. It regularly polls for new image versions from your container registry, checks if they meet the specified constraints, and updates your applications automatically.

Argo CD Image Updater also offers a range of advanced features, such as support for Helm and Kustomize-based applications, various update strategies (like semver, latest, name, and digest), and seamless integration with private container registries. Additionally, it allows parallel updates and supports filtering tags with custom matchers, making it highly customizable and suitable for both small and large-scale Kubernetes environments.

Use Cases and Problems Solved

Keeping Images Up-to-Date Automatically: For teams deploying frequently updated images, manually tracking and updating these images can lead to inefficiencies and downtime. Argo CD Image Updater solves this by automating the entire image update process.
Version Control and Stability: Sometimes teams want to stick to a specific versioning pattern like semver or ensure they only pick stable releases. With Argo CD Image Updater’s configurable strategies, it’s easy to enforce such policies and ensure the application is always running on the desired version.
Efficient GitOps Workflow: By integrating with Argo CD, changes can be written back to Git, maintaining the declarative model central to GitOps and ensuring that the application state and versioning are always in sync with the source of truth.

Whether you’re running a simple workload or managing complex deployments across multiple environments, Argo CD Image Updater provides a streamlined way to automate image updates, reduce operational overhead, and ensure that your applications are always running with the latest and most secure versions.

Configuration and Setup

In this example implementation, we are using the official argocd-image-updater helm chart, available at: https://github.com/argoproj/argo-helm/tree/main/charts/argocd-image-updater

It is deployed as an argocd application in the same cluster and namespace as Argo CD:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: argocd-image-updater
  namespace: argocd
spec:
  destination:
    namespace: argocd
    server: https://kubernetes.default.svc
  project: 'applications'
  source:
    helm:
      valueFiles:
      - ../argocd-image-updater/values.yaml
    path: helm/argocd-image-updater
    repoURL: https://gitlab.org.com/demo.git
    targetRevision: HEAD
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
      allowEmpty: false
    syncOptions:
  revisionHistoryLimit: 3

Let’s review the values file, where we’ll explore some of the essential configuration options required. These options are critical to ensuring proper functionality and deployment of the service.

Registries

Here we will configure the container registries that we are using. Argo CD Image Updater supports the majority of container registries (public and private), that implement Docker registry v2 API and has been tested against registries such as Docker Hub, Docker Registry v2 reference implementation (on-premise), Red Had Quay, Jfrog Artifactory, Github Container Registry, GitHub Packages Registry, GitLab Container Registry and Google Container Registry.

In the following examples, we will configure two of the most widely used container registries – Amazon Elastic Container Registry (ECR) and GitHub Container Registry (GHCR). In our case, we are working with private registries to ensure secure storage and access control for container images.

Amazon Elastic Container Registry (ECR) configuration:

registries:
  - name: ECR
    api_url: https://000000000000.dkr.ecr.eu-west-1.amazonaws.com
    prefix: 000000000000.dkr.ecr.eu-west-1.amazonaws.com
    ping: yes
    insecure: false
    credentials: ext:/scripts/login.sh
    credsexpire: 10h

For Amazon Elastic Container Registry, authentication is possible through a script that executes an API call to retrieve the necessary credentials. In the values file, we can include this script in the authScripts section:

authScripts:
  # -- Whether to mount the defined scripts that can be used to authenticate with a registry, the scripts will be mounted at `/scripts`
  enabled: true
  # -- Map of key-value pairs where the key consists of the name of the script and the value the contents
  scripts:
    login.sh: |
      #!/bin/sh
      aws ecr --region "eu-west-1" get-authorization-token --output text --query 'authorizationData[].authorizationToken' | base64 -d

The script is executed by the pod and is responsible for obtaining the ECR authorization token. We use a role attached to our EKS node group, which includes the AWS-managed policy AmazonEC2ContainerRegistryReadOnly. This policy permits the GetAuthorizationToken API call:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ecr:GetAuthorizationToken",
                "ecr:BatchCheckLayerAvailability",
                "ecr:GetDownloadUrlForLayer",
                "ecr:GetRepositoryPolicy",
                "ecr:DescribeRepositories",
                "ecr:ListImages",
                "ecr:DescribeImages",
                "ecr:BatchGetImage",
                "ecr:GetLifecyclePolicy",
                "ecr:GetLifecyclePolicyPreview",
                "ecr:ListTagsForResource",
                "ecr:DescribeImageScanFindings"
            ],
            "Resource": "*"
        }

GitHub Container Registry configuration:

registries:
  - name: GitHub Container Registry
    api_url: https://ghcr.io
    prefix: ghcr.io
    ping: yes
    credentials: secret:argocd/ghcr-secret#token

For registry authentication, in the credentials section, we are using Kubernetes secret. The #token part refers to the specific key (usually containing a personal access token or authentication token) inside the secret. The token must have at least read:packages permissions. Here is a manifest of the Kubernetes secret which has to be applied in the argocd namespace:

apiVersion: v1
kind: Secret
metadata:
  name: ghcr-secret3
  namespace: argocd
stringData:
  token: user_name:access_token

Enabling the service account and RBAC creation:

rbac:
  # -- Enable RBAC creation
  enabled: true

serviceAccount:
  # -- Specifies whether a service account should be created
  create: true
  # -- Annotations to add to the service account
  annotations: {}
  # -- Labels to add to the service account
  labels: {}
  # -- The name of the service account to use.
  # If not set and create is true, a name is generated using the fullname template
  name: ""

ServiceAccount provides the necessary identity for ArgoCD Image Updater to authenticate and interact with the Kubernetes API in order to perform updates on deployment manifests or Helm charts (e.g., changing container image tags).

rbac ensures that ArgoCD Image Updater is granted only the permissions it needs, helping to secure your cluster by restricting its access and reducing the attack surface.

Without enabling both, the ArgoCD Image Updater would either lack the permissions to modify Kubernetes resources (failing to update your applications) or could have overly broad permissions, which could be a security risk.

In the default installation scenario, i.e. Argo CD Image Updater installed to the argocd namespace, no further configuration has to be done for Argo CD Image Updater to access the Kubernetes API. If your Argo CD installation is in a different namespace than argocd, you would have to adapt the RoleBinding to bind to the ServiceAccount in the correct namespace.

Log Level:

# -- Argo CD Image Update log level
  logLevel: "debug"

Changing the log level from “info” to “debug” in the Argo CD Image Updater values file can be beneficial in certain scenarios where you need deeper insights into the system’s behavior.

Argo CD Image Updater binary:

The argocd-image-updater binary and specifically the test subcommand provides a variety of test options including testing registry access, multi-arch images, semver constrains, update strategies, and credentials before configuring annotations on your Argo CD applications. It is available in the argocd-image-updater pod or you can install it locally. Here are the argocd-image-updater test command options:

Flags:
      --allow-tags string             only consider tags in registry that satisfy the match function
      --credentials string            the credentials definition for the test (overrides registry config)
      --disable-kubernetes            whether to disable the Kubernetes client
      --disable-kubernetes-events     Disable kubernetes events
  -h, --help                          help for test
      --ignore-tags stringArray       ignore tags in registry that match given glob pattern
      --kubeconfig string             path to your Kubernetes client configuration
      --loglevel string               log level to use (one of trace, debug, info, warn, error) (default "debug")
      --platforms strings             limit images to given platforms (default [linux/amd64])
      --rate-limit int                specificy registry rate limit (overrides registry.conf) (default 20)
      --registries-conf-path string   path to registries configuration
      --semver-constraint string      only consider tags matching semantic version constraint
      --update-strategy string        update strategy to use, one of: semver, latest) (default "semver")

Update methods

Argo CD Image Updater supports two write-back methods for propagating new image versions to Argo CD:

argocd : Directly modifies the Argo CD application resource via Kubernetes or the Argo CD API, depending on the configuration.
git : Creates a Git commit in the application’s repository with the updated image information.

The write-back method and its configuration are set per application, with further configuration options available depending on the method used.

In this article, the examples are applied using the argocd update method, which is the default update method and does not need further configuration. For production environments, it is recommended to use the git update method to persist the changes made by Argo CD Image Updater in your git repository.

Update strategies

An update strategy specifies how Argo CD Image Updater identifies new image versions for updates. It supports various strategies for tracking and updating configured images. Each image can have its update strategy, with the default being the semver strategy.

The currently supported update strategies are:

• semver: Updates based on semantic versioning.

• latest: Updates to the most recently built image in the registry.

• digest: Updates to the latest version of a tag using its SHA digest.

• name: Sorts tags alphabetically and updates to the highest version

In the examples below we show how to annotate our argocd applications in order to enable Argo CD Image Updater, setting up all update strategies. We are using an umbrella helm chart to deploy our sample application. For Helm applications with multiple images in the manifest or when parameters other than image.name and image.tag are used to define images, you need to configure an <image_alias> in the image specification. This alias helps identify the image and enables the Ago CD Image Updater:

argocd-image-updater.argoproj.io/image-list: "<image_alias>=<some/image>"

semver update strategy :

This is the default update strategy. Via semver strategy Argo CD Image Updater operates with images tagged in semantic versioning format. Tags should include semver-compatible identifiers in the structure X.Y.Z, where X, Y, and Z are whole numbers. An optional prefix of “v” (for example, vX.Y.Z) can be used, and both formats are considered equivalent. In this first example each annotation is specifically explained because we are using some of the annotations for semver update strategy in all examples.

Example annotations:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: sampleapp
  namespace: argocd
  annotations:
    argocd-image-updater.argoproj.io/image-list: "sampleapp=0.dkr.ecr.eu-west-1.amazonaws.com/sampleapp:v1.2.x"
    argocd-image-updater.argoproj.io/sampleapp.helm.image-name: "sampleapp.deployment.image.repository"
    argocd-image-updater.argoproj.io/sampleapp.helm.image-tag: "sampleapp.deployment.image.tag"
    argocd-image-updater.argoproj.io/sampleapp.update-strategy: "semver"
    argocd-image-updater.argoproj.io/pull-policy: Always
    argocd-image-updater.argoproj.io/write-back-method: argocd

image-list – as we explained earlier, the image-list annotation enables Argo CD Image Updater to operate with the application – for the value we are using sampleapp as alias and we are specifying the image and its tag.

image-name – we are specifying the image name via its helm values path, where we are defining the image repository

image-tag – defines the image tag via its helm values path

update-strategy – here we are declaring the desired update strategy

pull-policy – specifying the pull-policy, in this case we are always getting the latest version.

write-back-method – specifying the Argo CD Image Updater write-back-method

In this scenario, we are using a semantic versioning constraint with the tag v1.2.x. This means that Argo CD Image Updater will look for any image tag that matches the v1.2.x pattern. The x in semantic versioning acts as a wildcard, so the updater will accept any patch-level version within the v1.2 series (e.g., v1.2.1, v1.2.5, v1.2.9, etc.).

Here is part of the helm values file that we are using for the sampleapp which is connected to the annotations:

sampleapp:
  appId: sampleapp
  deployment:
    enabled: true
    image:
      repository: "000000000000.dkr.ecr.eu-west-1.amazonaws.com/sampleapp"
      tag: "v1.2"
      digest: true
      pullPolicy: "Always"

latest update strategy:

Argo CD Image Updater can update the image with the most recent build date, even if the tag is arbitrary (like a Git commit SHA or random string). It focuses on the build date, not when the image was tagged or pushed to the registry. If multiple tags share the same build date, the updater sorts the tags in descending lexical order and selects the last one.

Example annotations:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: sampleapp
  namespace: argocd
  annotations:
    argocd-image-updater.argoproj.io/image-list: "sampleapp=0.dkr.ecr.eu-west-1.amazonaws.com/sampleapp"
    argocd-image-updater.argoproj.io/sampleapp.helm.image-name: "sampleapp.deployment.image.repository"
    argocd-image-updater.argoproj.io/sampleapp.update-strategy: "latest"
    argocd-image-updater.argoproj.io/pull-policy: Always
    argocd-image-updater.argoproj.io/write-back-method: argocd

In this scenario, we don’t have to specify image-tag. But if we want to allow only particular tags for update we can use the argocd-image-updater.argoproj.io/myimage.allow-tags: annotation, for example with latest and master tags:

argocd-image-updater.argoproj.io/myimage.allow-tags: latest, master

or we can ignore them with the ignore-tags annotation:

argocd-image-updater.argoproj.io/myimage.ignore-tags: latest, master

Here is part of the helm values file that we are using for the sampleapp which is connected to the annotations:

sampleapp:
  appId: sampleapp
  deployment:
    enabled: true
    image:
      repository: "000000000000.dkr.ecr.eu-west-1.amazonaws.com/sampleapp"
      tag: "latest" #in this case tag will be ignored
      digest: true
      pullPolicy: "Always"

digest update strategy:

This update strategy monitors a specified tag in the registry for any changes and updates the image when a difference from the previous state is detected using the image SHA digest. The tag must be defined as a version constraint in the image list. It’s ideal for tracking mutable tags like the latest or environment-specific tags (e.g., dev, stage, prod) generated by a CI system.

Example annotations:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: sampleapp
  namespace: argocd
  annotations:
    argocd-image-updater.argoproj.io/image-list: "sampleapp=0.dkr.ecr.eu-west-1.amazonaws.com/sampleapp:latest"
    argocd-image-updater.argoproj.io/sampleapp.helm.image-name: "sampleapp.deployment.image.repository"
    argocd-image-updater.argoproj.io/sampleapp.helm.image-tag: "sampleapp.deployment.image.tag"
    argocd-image-updater.argoproj.io/sampleapp.update-strategy: "digest"
    argocd-image-updater.argoproj.io/pull-policy: Always
    argocd-image-updater.argoproj.io/write-back-method: argocd

Here is part of the helm values file that we are using for the sampleapp which is connected to the annotations – the important thing here is to specify the image tag in the format –

tag: “tag_name@sha256” :

sampleapp:
  appId: sampleapp
  deployment:
    enabled: true
    image:
      repository: "000000000000.dkr.ecr.eu-west-1.amazonaws.com/sampleapp"
      tag: "latest@sha256:ef8049179764ee395542a9895dbc3e326b6526116672aea568cfb0a33c0912af"
      digest: true
      pullPolicy: "Always"

name update strategy:

This updated strategy sorts image tags lexically in descending order and selects the last tag for updating. It’s useful for tracking images using calver versioning (e.g., YYYY-MM-DD) or similar tags. By default, all tags in the repository are considered, but you can configure it to limit which tags are eligible for updates.

Example annotations:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: sampleapp
  namespace: argocd
  annotations:
    argocd-image-updater.argoproj.io/image-list: "sampleapp=0.dkr.ecr.eu-west-1.amazonaws.com/sampleapp:latest"
    argocd-image-updater.argoproj.io/sampleapp.helm.image-name: "sampleapp.deployment.image.repository"
    argocd-image-updater.argoproj.io/sampleapp.update-strategy: "name"
    argocd-image-updater.argoproj.io/myapp.allow-tags: regexp:^[0-9]{4}-[0-9]{2}-[0-9]{2}-stable$
    argocd-image-updater.argoproj.io/pull-policy: "Always"
    argocd-image-updater.argoproj.io/write-back-method: "argocd"

In this case, if we have tags such as: 2024-09-30-stable, 2024-09-30-beta, 2024-10-01-beta, 2024-10-01-stable, master, latest – Argo CD Image Updater will consider only the “-stable” ending tags, sort them lexically and choose the 2024-10-01-stable tag for the update.

Here is part of the helm values file that we are using for the sampleapp which is connected to the annotations:

sampleapp:
  appId: sampleapp
  deployment:
    enabled: true
    image:
      repository: "000000000000.dkr.ecr.eu-west-1.amazonaws.com/sampleapp"
      tag: "2024-09-30-stable" #will be ignored in this case
      digest: true
      pullPolicy: "Always"

After we’ve made the needed configurations and selected the most suitable update strategy we can check the Argo CD application’s parameters through the UI:

As we can see, after the new image version was pushed in ECR, the original value of the image tag was changed by the Argo CD image updater, and the new image was deployed!

In conclusion, the Argo CD Image Updater is a powerful tool that enhances the continuous delivery process in Kubernetes environments. Automating the process of updating container images, not only streamlines deployments but also reduces the risk of human error associated with manual updates.

Moreover, its flexibility allows developers to tailor the update policies to suit their specific workflows, ensuring that only the necessary updates are applied. This ultimately leads to improved application reliability and performance.

Reference: https://argocd-image-updater.readthedocs.io/en/stable/

Salt Lake City, United States

Understanding Argo CD Image Updater

Use Cases and Problems Solved

Configuration and Setup

Update methods

Update strategies