Guest post by Zerto
Gone are the days that a backup of a virtual machine is enough for backing up an entire application.
Applications, especially modern, cloud-native applications, span across virtual machines, containers and cloud services. Backing up any one of those individual components is not enough to capture the entire application’s state. Instead, we need to capture all application components at the same moment in time for the backup to be usable. In other words: if we don’t capture the entire application, we cannot meaningfully recover data from a backup.
What is Application Consistency
Application Consistency is the practice of capturing the entire application’s state as a whole, coordinating backup across its constituents across virtual machines, containers and cloud services.
Modern, cloud-native applications, by design, are made up of many smaller application services, called microservices. These services live across cloud availability zones (or span data centers) by design to increase resilience.
This makes application consistency for modern applications more complex than ever — the number of components and diversity of storage locations is higher than ever.
In the case Kubernetes, application consistency is even more important – and challenging. There are two significant contributing factors to this challenge.
Containers are ephemeral
The first is that containers are ephemeral – you can’t rely on any one single container running. The sum of running containers make up that application service; individual container instances pop in and out of existence during normal operations, caused by scaling events (adding or removing replicas) and host events (host failures, cluster events).
This means that, for data protection purposes, we cannot use the containers themselves to capture state. We have to rely on the underlying Kubernetes platform, understand the taxonomy of the application, and use its knowledge of the application to capture container images, application configuration and state and persistent storage.
Luckily, application taxonomy is built into the core of Kubernetes (we won’t go into detail on any of the core Kubernetes objects, but you can go here for a great explanation). Using Deployments, ReplicaSets and Pods, you can tell Kubernetes what your application looks like, exactly. And in combination with Services, Ingress, ConfigMaps, Secrets and some more object types, Kubernetes unravels applications to its individual constituents, which we can use for application consistency.
For instance, this yaml is an example of a Deployment. It creates a ReplicaSet to bring up three nginx Pods. The Deployment, called nginx-deployment, creates three replicated Pods, each consisting of a single container, based on the nginx Docker Hub image at version 1.14.2.
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80Source: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/
As you can see, Kubernetes is very prescriptive. The desired state approach of Kubernetes, where you define what you want your applications to look like, is helpful for application consistency: the entire taxonomy of an application is available, in code, ready to use. Kubernetes cleanly separates the application’s container base images, application configuration and state, secrets and persistent storage.
Stateless containers lead to fragmentation
Which brings us to the second significant contributing factor to the Kubernetes application consistency challenge: containers are stateless, and lead to fragmentation of persistent data.
Kubernetes cleanly separates application runtime (the container image), application configuration (ConfigMaps, persistent state on disk) and application data (persistent volumes). That means that if we want to capture an application’s state, we need to capture more than the container images, but also need to protect additional Kubernetes objects (like ConfigMaps, Secrets, Services and more), but also its persistent data across on-premises and cloud block storage, but also persistent data on file and object stores at the same point-in-time.
Which, inevitably, leads to data fragmentation across heterogenous storage services and locations, including across availability zones and on-prem data centers. Keeping tabs on what data is stored where (and how) is not easy, adding to the complexity of application consistency of modern applications.
This is why it’s important to keep state in one place: the Kubernetes yaml, so you can treat that yaml as your single source of truth (as well as its history of changes) to an application’s anatomy. With that visibility into your applications landscape’s state, you can consistently, correctly and completely protect its data.
Application-level Consistency Groups
With your application yamls being the truth, the whole truth and nothing but the truth, application consistency suddenly becomes a trivial issue to solve. After all, we now know everything we need to know about the application to be able to consistently back it up across the entirety of its constituents. This allows us to protect the entire application, its metadata, configuration, state and persistent data in one fell swoop at the same point-in-time.
This allows us to protect and recover them as a one consistent and cohesive entity along with all their objects and metadata, such as ConfigMaps, StatefulSets, Services, and Deployments.
However, due to the complexity of these applications, even a single snapshot across all of these does not guarantee a consistent single point-in-time protection (and subsequently, recovery), especially in applications using heterogeneous storage services due to differences in the underlying snapshotting technologies in each service.
In order for the entire application to be consistently protected, it needs a more granular approach than snapshots or agent-based backups. Journaling has a granularity of seconds, which ensures you can rewind the entire application (including the persistent data) to any previous point in time. If that point is not satisfactory, just discard it and pick another one, it is truly that easy. Journaling has the benefit of write-order fidelity across multiple present volumes, which makes sure that any changes to the entire application regardless of its microservices architecture are protected in the same order, guaranteeing protected data is consistent with the production application on a second-to-second basis.
If you want to experience application-level consistency for Kubernetes-based workloads yourself, check out the hands-on lab to explore application consistency first-hand in a controlled environment with no additional setup or configuration required.