SysEleven opted for Cilium to simplify networking, enhance observability
Challenge
SysEleven is a cloud service provider offering managed and unmanaged cloud services to customers across Europe. They also offer a managed Kubernetes product called MetaKube.
SysEleven initially built the networking layer of its cloud platform using the Canal project, which combines the Calico policy engine with Flannel’s command-line interface (CLI) tool. However, they encountered issues securing their pods. Over time, the setup’s increasing complexity made it difficult to configure for their specific use case.
Solution
SysEleven selected Cilium as their networking, security, and observability solution for their cloud platform due to its simplicity, native routing capabilities, and prominence in the cloud native ecosystem.
Impact
Migrating their cloud platform’s internal and external clusters from Canal to Cilium, powered by eBPF (extended Berkeley Packet Filter), allowed SysEleven to boost network performance and also enhance insight with Hubble. They’ve also enhanced user experience by serving customers who specifically requested Cilium as part of their cloud service.
Projects used
By the numbers
450
Clusters
4,000
Network policies
200%
Through-put improvement (pod to pod)
Migrating to Cilium for Simplified Cloud Native Networking
SysEleven’s cloud and Kubernetes platform team operates its cloud infrastructure, which includes multiple data centers across Germany. This infrastructure supports around 400 Kubernetes clusters belonging to various customers on their OpenStack cloud.
They initially built their managed Kubernetes service using Canal for networking. However, they encountered increasing complexity when configuring the tool to meet their needs. Additionally, they faced issues with pod security and were unable to perform native routing.
“Before we started using Cilium, we were using Canal. We initially chose to use this CNI plugin because it was simple to set up. Unfortunately, how we set this up didn’t allow us to solve some issues and was becoming increasingly complex to maintain.”
Simon Pearce, Kubernetes & Cloud Consultant, SysEleven
Faced with these challenges, the SysEleven team decided to seek a better solution. During their evaluation process, they prioritized simple configuration and the use of eBPF. Ultimately, they had to choose between Calico and Cilium.
“We first discovered native pod routing and realized that Cilium could provide it to solve our issues. Once we were considering changing, we thought we might as well review all other aspects of the CNI to see if there was something better. We were looking at Calico with eBPF and Cilium as the only real competitors in the space because both had a large community and user base around them.”
Josef Mende, Technical Lead & Product Owner for MetaKube, SysEleven
SysEleven chose Cilium as its networking solution due to its ease of use and configuration and its utilization of eBPF.
“What stood out with Cilium for us was its easy configuration; it is built for Kubernetes and utilizes eBPF. You don’t have so much noise in the configuration, which was where we had some trouble with Calico. Over time, Calico introduced more and more features, and then there were suddenly many different ways of configuring it because of all these new parameters. We preferred Cilium because of this.
When we tested Cilium, we never had issues. We installed it with a Helm chart and only needed to change eight or ten lines in the values.yaml for Helm, and that’s all the configuration we needed. With Calico, it was much, much more. Cilium’s simplicity was a strong reason for us to choose it.”
Josef Mende, Technical Lead & Product Owner for MetaKube, SysEleven
With the choice made, they installed Cilium via Helm and began migrating smaller clusters to it for internal and customer workloads. Use of Cilium then spread to their networking stack and also helped them replace Kube-Proxy.
“Cilium replaced everything that previously had anything to do with networking. In one sense, it’s just a CNI plugin, but on the other hand, it can also remove the need for so many other tools, like Kube-Proxy.”
Josef Mende, Technical Lead & Product Owner for MetaKube, SysEleven
Once SysEleven switched to Cilium’s native routing, they also experienced a 200% increase in network throughput and enhanced observability.
“Native routing also allows us to give customers the option to use Cilium’s observability component, Hubble. This makes it much simpler for customers to debug their own Kubernetes clusters. With IP encapsulation in overlay routing, finding issues is difficult because you have to unpack all these IP-encapsulated packets and look into them. Hubble and native routing make things a lot easier.”
Simon Pearce, Kubernetes & Cloud Consultant, SysEleven
Better Observability and Troubleshooting with Hubble
Deeper observability has been one of the top benefits of migrating to Cilium for SysEleven and their customers. Internally, they use Hubble to monitor their network traffic and help them debug network issues. They also offer their customers Hubble as an observability add-on for self-service network debugging, saving both sides time.
“Cilium gave us better insight into what’s happening in the CNI layer through Hubble and its CLI and UI. I prefer the CLI tool best because it gives me the most output and saves me from using many other tools I needed before when debugging containers. Hubble makes troubleshooting a lot easier.”
Simon Pearce, Kubernetes & Cloud Consultant, SysEleven
“Hubble makes debugging easy because you trust what it tells you; you know what happened and where to look. For that, we are very grateful that it exists. For our customers, we install Hubble Relay and UI. This means that they can troubleshoot things by themselves and not rely on us. If customers don’t have much network knowledge, they can’t use Wireshark or something similar. Being able to use Hubble to solve their issues is a huge win for everyone.”
Josef Mende, Technical Lead & Product Owner for MetaKube, SysEleven
Providing Better Customer Experience with Cilium
Migrating to Cilium has been a success for SysEleven, enabling them to improve network performance, observability, and also user experience for customers across experience levels.
“We have one side of our customers who don’t care much about what CNI they’re running; they just care that their cluster is very stable and works well. Cilium makes them happy by providing a faster, smoother, and better user experience than before. We also have quite a few experienced and advanced users who are already familiar with Cilium and work with it. This migration has been a game changer for us because they’ve said, ‘Okay, Cilium is our way to go forward, and we’re pleased that you made this choice too. We’re glad to work with you because you’re using Cilium.’ I would say for a couple of the new customers that recently joined us, Cilium was an important reason why.”
Simon Pearce, Kubernetes & Cloud Consultant, SysEleven
The SysEleven team is very excited about Cilium and plans to explore other technologies in its ecosystem, including Tetragon, BGP, Gateway API, and Cilium cluster mesh.
“Some exciting things are coming around, like Tetragon for security; that’s something we will look at internally. We would also like to use BGP, which could allow us to mesh clusters together with cluster mesh.
I also like the idea that Cilium has integrated ingress with Envoy, and started integrating the Kubernetes Gateway API, because it allows us to remove another abstraction layer. You only need to install the CNI and all of a sudden you get the CNI networking component, network policies, observability, and L7 traffic on top with just one piece of software instead of having to install four or five different pieces of software.”
Simon Pearce, Kubernetes & Cloud Consultant, SysEleven
“I think almost every feature that Cilium has is very interesting because it’s implemented in the right place – that’s why it’s so good. If something can be done with Cilium versus something else, then we will choose Cilium.”
Josef Mende, Technical Lead & Product Owner for MetaKube, SysEleven