Guest post originally published on InfraCloud’s blog by Sahil Lakhwani, Software Engineer at InfraCloud
It’s been about a year when we wrote about Crossplane. The previous post explains how Crossplane helps you provision and manage infrastructure using the Kubernetes API. Crossplane has since evolved and reached v1.0! Big congratulations to the Crossplane community!
Besides being an interface to cloud providers, Crossplane now has mechanisms to create and publish your own custom resources, without writing code. It offers ways to define your own infrastructure resources declaratively, in form of CRDs on top of cloud providers APIs. In this post, we’ll see how you can use Crossplane to create your own control plane on top of cloud providers.
Infrastructure Provisioning
Let’s take a look at how Crossplane allows us to provision cloud resources.
Installation
The below commands will install Crossplane in a Kubernetes cluster using Helm 3
$ kubectl create namespace crossplane-system
$ helm repo add crossplane-stable https://charts.crossplane.io/stable
$ helm repo update
$ helm install crossplane --namespace crossplane-system crossplane-stable/crossplane
Crossplane CLI is kubectl plugin to manage Crossplane packages. To install, run:curl -sL https://raw.githubusercontent.com/crossplane/crossplane/release-1.0/install.sh | sh
Working with a Cloud Provider
A “provider” in Crossplane is a bunch of CRDs and their controllers, creating an interface to a cloud provider API (in fact, any API). We’ll use AWS for demonstration here.
Install the providerkubectl crossplane install provider crossplane/provider-aws:v0.16.0
Create a provider secret from the AWS credentials by running the following. This uses the default profile from AWS CLI credentials.
$ AWS_PROFILE=default && echo -e "[default]\naws_access_key_id = $(aws configure get aws_access_key_id --profile $AWS_PROFILE)\naws_secret_access_key = $(aws configure get aws_secret_access_key --profile $AWS_PROFILE)" > creds.conf
$ kubectl create secret generic aws-creds -n crossplane-system --from-file=key=./creds.conf
And a ProviderConfig to configure credentials for the AWS provider:
$ cat > providerconfig.yaml <<EOF
apiVersion: aws.crossplane.io/v1beta1
kind: ProviderConfig
metadata:
name: default
spec:
credentials:
source: Secret
secretRef:
namespace: crossplane-system
name: aws-creds
key: key
EOF
$ kubectl apply -f providerconfig.yaml
Below is a Crossplane representation of a AWS RDS instance, called a “managed resource”. A Crossplane managed resource is a representation of a cloud provider resource and in fact, any API.
$ cat > rds.yaml <<EOF
apiVersion: database.aws.crossplane.io/v1beta1
kind: RDSInstance
metadata:
name: rdspostgresql
spec:
forProvider:
region: us-east-1
dbInstanceClass: db.t2.small
masterUsername: masteruser
allocatedStorage: 20
engine: postgres
engineVersion: "9.6"
skipFinalSnapshotBeforeDeletion: true
writeConnectionSecretToRef:
namespace: crossplane-system
name: aws-rdspostgresql-conn
EOF
$ kubectl apply -f rds.yaml
This will create an RDS instance and a Kubernetes Secret with the credentials for it.
Infrastructure Composition
While managed resources are great for managing cloud resources, it can soon become overwhelming to manage lots of them each time. For instance, an application developer might not be interested in the details of how a RDS instance is being created and managed with firewall rules, database configuration, etc. The only thing of interest might be a database and credentials to access it. On the contrary, an infrastructure operator may not want to expose all of the RDS configurations to a developer, but instead restrict only to some necessary parameters.
Crossplane offers mechanisms to compose managed resources where users can create their own abstractions in a declarative way.
Composite Resources: A Composite Resource (XR) is a custom resource that is composed of managed resources allowing you to abstract the infrastructure details. A CompositeResourceDefinition (XRD) defines a new kind of composite resource. XRDs are cluster scoped. To create a namespaced XR, the respective XRD may optionally offer a Composite Resource Claim (XRC).
Composition: A Composition specifies which resources a XR will be composed of i.e what happens when you create XR. There can be multiple compositions for one XR. For example, for a CompositeDatabase XR, you may have a composition which will create a AWS RDS instance, a security group and a MySQL database. Another composition can define GCP CloudSQL instance and a PostgreSQL database.
Configuration: A configuration is a package of XRDs and Compositions that can be then published using the Crossplane CLI to an OCI image registry and installed into a Crossplane cluster by creating a declarative Configuration resource
The below figure shows how the Crossplane resources might look like in a Kubernetes cluster. The XRDs are cluster scoped with compositions satisfying them. These XRDs offer a XRC each, which are namespace scoped. Offering a XRC is optional and is done to allow users to create XRs in their own namespace.
The XRDs also are referring to a managed resource in the cluster scope, so that the XRC can use it.
Let’s see a quick example to understand these concepts better. We’ll create a CompositePostgreSQLInstance XRD for PostgreSQL database , a composition for satisfying this XRD, and see how we can use the XRC that gets created.
You can leverage the Crossplane CLI to install a configuration package:kubectl crossplane install configuration registry.upbound.io/xp/getting-started-with-aws:v1.0.0
Otherwise, you can define the XRD and composition yourself as explained below. (The examples are taken from official Crossplane docs)
The XRD is defined below. Note how a claim is offered under claimNames. The only configurable field in the spec is storageGB
// xrd.yaml
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
name: compositepostgresqlinstances.database.example.org
spec:
group: database.example.org
names:
kind: CompositePostgreSQLInstance
plural: compositepostgresqlinstances
claimNames:
kind: PostgreSQLInstance
plural: postgresqlinstances
connectionSecretKeys:
- username
- password
- endpoint
- port
versions:
- name: v1alpha1
served: true
referenceable: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
parameters:
type: object
properties:
storageGB:
type: integer
required:
- storageGB
required:
- parameters
The composition defined below defines creation of a AWS RDS instance. Notice how it references the XRD using the compositeTypeRef field. The resources define a set of base resources that get created when this composition is used. And under that, the patches field is used to fetch the property values from the XRD.
// composition.yaml
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
name: compositepostgresqlinstances.aws.database.example.org
labels:
provider: aws
guide: quickstart
vpc: default
spec:
writeConnectionSecretsToNamespace: crossplane-system
compositeTypeRef:
apiVersion: database.example.org/v1alpha1
kind: CompositePostgreSQLInstance
resources:
- base:
apiVersion: database.aws.crossplane.io/v1beta1
kind: RDSInstance
spec:
forProvider:
region: us-east-1
dbInstanceClass: db.t2.small
masterUsername: masteruser
engine: postgres
engineVersion: "9.6"
skipFinalSnapshotBeforeDeletion: true
publiclyAccessible: true
writeConnectionSecretToRef:
namespace: crossplane-system
patches:
- fromFieldPath: "metadata.uid"
toFieldPath: "spec.writeConnectionSecretToRef.name"
transforms:
- type: string
string:
fmt: "%s-postgresql"
- fromFieldPath: "spec.parameters.storageGB"
toFieldPath: "spec.forProvider.allocatedStorage"
connectionDetails:
- fromConnectionSecretKey: username
- fromConnectionSecretKey: password
- fromConnectionSecretKey: endpoint
- fromConnectionSecretKey: port
Install these two manifests
$ kubectl apply -f xrd.yaml && kubectl apply -f composition.yamlYou can now see a new XRC created, postgresqlinstances.database.example.org. Use this XRC to create a AWS RDS instance.
// xr.yaml
apiVersion: database.example.org/v1alpha1
kind: PostgreSQLInstance
metadata:
name: my-db
namespace: default
spec:
parameters:
storageGB: 20
compositionSelector:
matchLabels:
provider: aws
vpc: default
writeConnectionSecretToRef:
name: db-conn
$ kubectl apply -f xr.yaml
This will result in creation of RDS PostgreSQL instance. Similarly, you can have another Composition for GCP CloudSQL and create a XR by modifying the compositionSelector field in the above manifest.
Creating Control Planes
As we saw, Crossplane helps to build custom resources on top of cloud resources and in fact on top of any API. This is great to build a control plane with required abstractions and restrictions using XRD, composition and XRC.
The below depiction shows an example of how an infrastructure operator might allow other people to create their database instances which are attached to VPCs which they don’t have control of.
The image shows a VPC at cluster scope. Also at the cluster level are two database XRDs offering a claim each. Other users can then create their own database XRs using the namespace scoped XRC and still refer to the VPC. There can be multiple XRDs with different configurations on the cluster scope, allowing users to choose from. Also, different compositions for different cloud providers to satisfy those XRDs.
The defined resource definitions can then be packaged and distributed as a configuration. You can read more how the Crossplane CLI helps you do this.
Conclusion
Crossplane helps to provision infrastructure using the Kubernetes API. Moreover, it helps to build custom abstractions and restrictions over any API. This makes it a great tool to build a single control plane to manage multiple cloud providers.
We hope this post gives you a good start for using Crossplane. We’re always thrilled to connect to people working with cloud native technologies. You can reach out to us via Twitter and LinkedIn.