Using Network Policy to Build Data Sandboxes at Bloomberg
Challenge
Bloomberg is a technology company that builds products for professionals in the finance industry. The company works to provide financial information and data to its customers through the Bloomberg Terminal, enterprise products and data feeds, and its media and news channels. Bloomberg offers its clients a next-generation quantitative investment solution known as BQuant Enterprise. Designed for quantitative analysts, it allows the rest of the financial industry to seamlessly integrate their coding workflows with Bloomberg’s data and tooling.
Data security is one of Bloomberg’s top priorities. As the team built out the data sandboxes for the BQuant Enterprise cloud, they sought a solution that could effectively bolster the protection of the firm’s and its clients’ data, focused on restricting automated data egress and customer resource access.
Solution
The Bloomberg Engineering team in San Francisco that is responsible for building this solution turned to Cilium to take advantage of its host-based network policies and its ability to easily replace the basic cloud CNI they were using. Cilium’s host-based policies helped the team restrict cluster network access to specific ports and host names to restrict data egress. After deployment, when it was time for the team to build their own data sandbox storage solution, Cilium also made it easier for them to create an access control system for the data from user workloads via its policy exception feature. Hubble also provided better observability and learning for the company’s Engineering teams and simplified network debugging.
Impact
By using Cilium, Bloomberg successfully enhanced the security and access control of its BQuant Enterprise workloads through the implementation of robust network security measures. This proactive approach has effectively prevented unauthorized egress of protected data or accessing of unauthorized resources. Bloomberg found a reliable method to enforce licensing restrictions across its datasets using Cilium’s network policies, thereby mitigating the risk of unauthorized access to sensitive information. Cilium’s Hubble also provides observability to troubleshoot network problems and build networking knowledge within Bloomberg’s teams, allowing its Engineers to save valuable development time.
Projects used
Enforcing Efficient Network Security with Cilium
Bloomberg offers a host of financial products to its clients, including BQuant Enterprise, which enables users to build, test, and share research internally within their financial institution as interactive applications, enabling a faster time to market.
The company has multiple Engineering teams working on this solution, and the main team that uses Cilium is focused on identity management, authentication, and connectivity for the solution.
Before implementing Cilium, the BQuant Engineering team recognized the importance of robust network security for the product they were offering to its clients. With a proactive mindset, the team sought ways to enhance network security measures and ensure a high level of data protection. They began to evaluate different tools and initially decided to use the cloud provider VPC CNI. However, it had several limitations, such as lower performance and a lack of professional support. It also lacked some of the advanced network policy features they were seeking.
“We started by looking at some other tools, and we first used [the cloud provider CNI]. But we found that Cilium, with its host-based policies and its ability to replace what we had out of the box, was really valuable.”
Anne Zepecki, Team Lead for the BQuant Enterprise Identity Management team in Bloomberg’s San Francisco Engineering Office
During the evaluation stage, the team also found Cilium to be the easiest to implement with their current infrastructure and ecosystem of tools, offering greater value with minimal effort required. They looked at benchmark numbers for Cilium’s eBPF data plane and were excited about the performance boost it could give them.
“When we were evaluating different options, we found Cilium was the least disruptive to implement in our tech stack. Other solutions require a sidecar running for every single container, but Cilium was a lot less lift to add and it provided us a lot of value right out of the box,” said Zepecki.
Optimizing the Infrastructure: Harnessing Layer 7 Policies and Hubble For Observability
With Cilium now a standard part of the BQuant Engineering team’s infrastructure, it became even more useful when they had to build complex data sandboxes and a data sandbox storage solution.
To create a storage solution for the data sandbox, Bloomberg’s Engineers needed to create an isolated environment that restricted and controlled how user data flows. They needed to enforce a Layer 7 network policy that prevented users from accessing protected data from other BQuant Enterprise workloads.
They implemented Cilium’s Layer 7 policies to block traffic from leaving the BQuant Enterprise platform, and they’ve also been able to add exceptions for different types of storage they want to be able to access from the workloads. With these in place, they were able to build out their data sandbox storage solution.
“When it was time for us to build a more explicit data sandbox and storage solution, we found it was a lot easier to do because we already had Cilium running as our CNI. It worked out super nicely and also unlocked the ability to offer additional features to our customers, like data replication,” said Zepecki.
Beyond just networking with Cilium, they also found Hubble to be very useful when troubleshooting. Hubble’s ability to visualize network calls and traffic flow between services was helpful to the application Engineers because they aren’t networking experts themselves. Hubble shortened the time it took for them to learn how the network functions, enabling the application teams to debug issues quickly.
“We’ve found that Hubble is a really beneficial tool to people who are new to the team to see and understand the network activity. It has been a powerful way to visualize all the pieces of our BQuant Enterprise platform. Since we’re supporting an enterprise product, we need to have a deep understanding of what’s happening and, if something were to go wrong, be able to investigate and troubleshoot it. Hubble has been really helpful in enabling us to visualize exactly what is happening in that workflow.”
Anne Zepecki
Aligning Business Needs and Delivering Added Value
Cilium is a major success story for the BQuant Enterprise team at Bloomberg, providing value and solving their network security and observability needs. After replacing their existing cloud provider CNI with Cilium, they were able to implement advanced network policies to protect their workloads and restrict customer access to unauthorized resources. Cilium also helped them easily secure a storage solution for their data sandbox. As an added bonus, Hubble now provides visibility for its application teams and helps them debug complex network problems.
“Don’t be afraid to start small and keep building. We introduced Cilium initially as our CNI, and we’ve been able to build a lot of really valuable functionality on top of it for our data sandbox and our sandbox storage functionality.”
Anne Zepecki
If you would like to hear more about Bloomberg’s use of Cilium in its BQuant Enterprise quant analytics platform, watch their talk at CiliumCon EU 2023.