Member post originally published on the Kubesimplify blog by Kunal Verma
With the increase in software complexity and as digital threats evolve (as discussed in our previous blog) in today’s digital age, the organizations are facing a massive increase in number of security vulnerabilities. According to the Coalition Cyber Threat Index 2024, we can expect a 25% rise in Common Vulnerabilities and Exposures (CVEs), reaching almost 34,888 this year – which is an astounding number!
Such a rise in CVEs presents a significant challenge for IT professionals who must constantly identify, prioritize, and remediate these vulnerabilities. This process, often pretty overwhelming, is commonly termed as “CVE Fatigue” and can lead to missed vulnerabilities and an increased risk of cyberattacks.
In this blog, we will delve into the primary causes of CVE fatigue and discuss some actionable strategies to minimize CVE fatigue. Thus, guiding organizations towards a more controlled and secure environment.
What is CVE Fatigue?
If you’re not already familiar with the term CVEs, it stands for Common Vulnerabilities and Exposures. These are standardized identifiers for known software and hardware vulnerabilities. The purpose of the CVE system is to provide a standardized way to identify and describe vulnerabilities, making it easier for organizations to prioritize and address security issues.
For more detailed information on CVEs, you can refer to the article.
CVE Fatigue on the other hand is a common term which is used to describe the overwhelming feeling experienced by developers and IT professionals due to the constant influx of new CVEs being introduced in the industry, that require immediate attention. This constant flow of discovering new vulnerabilities and quickly remediating them can lead to burnout and overlooked issues, increasing the risk of security breaches.
A notable incident that illustrates the consequences of CVE fatigue is the 2017 Equifax data breach. In this breach, attackers exploited a known vulnerability (CVE-2017-5638) in the Apache Struts framework. Although a patch for this vulnerability had been available for months, it was not applied in time, leading to the exposure of sensitive personal information of approximately 147.9 million individuals.
The Equifax breach is one of the prime examples of CVE fatigue because the company’s security team was likely overwhelmed by the sheer volume of vulnerabilities needing attention, which led to the delay in patching a critical vulnerability. This incident underscores the importance of effective vulnerability management and the dire consequences of CVE fatigue.
Did you know? According to the IBM Cost of a Data Breach Report 2023, the global average cost of a data breach in 2023 was $4.45 million, a 15% increase over three years. This certainly highlights the financial impact of CVE breaches and the importance of addressing CVE fatigue to prevent such costly incidents and most importantly – the customer’s trust!
CVE fatigue is a significant challenge for many organizations, arising from several key issues. By understanding the root causes and applying proven strategies, we can effectively mitigate this issue.
Use Minimal Dependencies
The sheer number of CVEs published each year can overwhelm security teams. With thousands of new vulnerabilities to assess and address, security teams are overwhelmed with alerts that require immediate attention. This high volume makes it difficult to keep up, leading to delayed responses and potential oversights.
Reducing the number of dependencies from the start can significantly decrease the number of potential vulnerabilities. For many companies and distributions, the common practice is to start with a general-purpose Linux image and strip away the things that aren’t needed.
Note: This is what DockerSlim does by taking an existing image and removing unnecessary components.
While this approach results in a smaller version of the main distribution, such as Ubuntu minimal, a more efficient way is to add only the necessary dependencies from the start. You can use minimal base images like Distroless for this purpose, but it isn’t straightforward to add or remove OS packages with this approach. Even a small dependency can cause a significant breach! And, if a developer doesn’t find a package they need in these minimal images, they often switch to much larger images instead of adding just the OS package they need.
Talos Linux, which includes only 12 binaries compared to other systems with hundreds, is a great example of this strategy, as its designed from scratch to include only what is needed. This approach limits the attack surface and reduces the burden on security teams.
Note: If you’re wondering how to have complete control over the OS packages your app needs, check out BuildSafe to get started.
Have a Dependency Update Mechanism
Effective management of software dependencies is critical in addressing CVE fatigue and minimizing vulnerabilities. Using a declarative format for dependencies helps eliminate implicit behavior problems. Declarative dependency management makes it clear which dependencies are being used, making it easier to track and update them. By explicitly declaring all dependencies in configuration files, teams can easily manage and automate updates, ensuring that all components are up-to-date and secure. Tools like Dependabot automate this process by checking for outdated dependencies and creating pull requests to update them.
However, it’s important to note that Dependabot cannot handle OS package updates because projects don’t typically have OS dependencies defined declaratively. This inconsistency makes it hard to track and update them, leading to unnoticed security vulnerabilities. Furthermore, very few package managers, such as Nix, declaratively define OS dependencies per project, making this an ongoing challenge in dependency management.
Note: The team at BuildSafe is working on a GitHub bot to address this issue for OS dependencies. If you’re interested, you can join the Discord server to request access to the alpha version!
Did you know? Repology, a project that tracks packaging information across many repositories, shows the fragmentation in package management. This fragmentation highlights how different repositories handle packages and the inefficiencies of using multiple package managers.
It’s worth noting that Nix stands out for having one of the largest package ecosystems with the least potential CVEs, highlighting its robustness and security!
Use a Single Package Manager and Automated Patching
Managing multiple package ecosystems can overwhelm security teams, leading to inefficiencies and delayed patching.
Enforcing the use of a single package manager across an organization simplifies dependency management. A single package manager provides a unified ecosystem, making it easier for security teams to track and update dependencies. For instance, when there’s a CVE out and the security team is responsible for patching mission-critical software, they often need to send patches to the upstream package repository. Imagine if you and your team had to do this for five different package ecosystems, each with its own build recipes and specific requirements, because of a single CVE – what a nightmare!
Ideally, as companies grow, the aim should be to have an internal package registry where dependencies are vetted by the security team before being used by developers. These internal repositories can mirror public ones while adding and maintaining additional packages with security checks and automated update strategies. This approach not only reduces the complexity and workload for security teams but also ensures that all dependencies are consistently managed and secure.
Thoughts: Imagine the benefits if we could share a package manager across both our development and runtime environments. Nix, a powerful package manager allows us to do this, but more on that in a future article.
For instance, Wolfi by Chainguard is a good example here. Wolfi is a distribution focused on being CVE-free and has achieved the status of Docker Trusted Publisher.
Prioritize Vulnerability Management
Determining which vulnerabilities to address first can be complex, as not all CVEs pose the same level of risk. The importance of the affected systems, potential impact, and severity need to be considered.
You can prioritize vulnerabilities by requesting Vulnerability Exploitability eXchange (VEX) documents from security advisories within your package manager ecosystem. These documents help eliminate bogus CVEs and provides accurate information about vulnerabilities. VEX documents can be generated using tools like Kubescape, which utilizes eBPF to monitor syscalls and determine the relevance of specific CVEs.
Note: If you’re still overwhelmed despite having minimal dependencies, it might be worth considering more complex solutions like the CVE Prioritizer. This project offers advanced methods to help manage and prioritize CVEs effectively.
Conclusion
Adopting a zero CVE strategy is crucial for securing your software, and it all starts with minimizing CVE fatigue. We’ve all heard that “Security shouldn’t be an afterthought” and to “Shift left”! It’s time to apply these principles in our software development processes by following the proven strategies we’ve outlined.
By reducing dependencies, keeping them up-to-date, using a single package manager, and effectively prioritizing vulnerabilities, you can mitigate the risks associated with CVE fatigue.
This proactive approach not only lightens the workload on security teams but also ensures that vulnerabilities are promptly addressed, maintaining a secure software environment. We urge you to check out projects like BuildSafe that aim to make security by default extremely easy. Let’s make security a fundamental part of our software development process and ensure our systems are robust and secure from the start.
