Effectively remediating vulnerabilities for all of your deployed software is a challenge. Vulnerabilities in container base images or open source dependencies you rely on may be discovered at any time in the lifecycle of a software component, not only when you scan for vulnerabilities during continuous integration builds in your development process. This page outlines some best practices for managing vulnerabilities in your software development systems when integrating with Apollo’s risk management features.
While you can leverage Apollo to identify impacted environments and proactively deploy fixes, you should also consider improvements to your entire software development lifecycle (SDLC). The following best practices are critical:
Maintaining a minimal dependency footprint is crucial for reducing vulnerabilities in your software. By using only necessary dependencies for your application to function effectively, you can minimize the risk of CVEs and ensure seamless deployment with security-focused CI/CD systems that enforce strict security controls.
Reducing dependency footprint can be challenging, but it is the most effective way to decrease the occurrence of new CVEs and overall CVE load. Investigating problem-prone dependencies and taking the effort to remove them can be worthwhile.
Below are some specific suggestions:
By following these suggestions and actively managing your dependency footprint, you can minimize the risk of vulnerabilities and maintain a more secure software environment.
The more software that is deployed together, the more likely it is that there is some vulnerability that needs remediation among all of those software components. If your security stance is to never upgrade to something that has security vulnerabilities, you run the risk of never being able to upgrade your software.
It is important to decouple your software components to allow for them to be individually secure or vulnerable. You do not want poor vulnerability management from a single team or software component to stop unrelated teams from making changes to their deployed software. The more teams that contribute to your software products, the more important this becomes.
Apollo supports this using Product dependencies, so that you can deploy multiple Helm charts that work together to build a larger application while still iterating quickly on each component independently.
Regularly updating dependencies and automating releases are essential for maintaining secure software. Automated dependency upgrades help proactively address vulnerabilities and reduce long-term costs. As your product count increases, you should rely on automation for dependency updates and frequent product releases.
You should use high-quality tests and features like blue/green upgrades, health checks, and rollbacks to ensure safe dependency upgrades. Address breaking changes early to minimize costs. Employ tools like GitHub's Dependabot or custom scripts for automation, and rebuild container images with the latest package versions during every release.
To maintain a secure software environment, you should establish and adhere to strict security SLAs for remediating vulnerabilities in deployed software. Rapid promotion and deployment are crucial for ensuring that new releases with vulnerability fixes are adopted quickly by users.
Addressing vulnerabilities in continuous integration (CI) alone is insufficient; the deployed versions of your software must also be upgraded. This can be particularly challenging for hard-to-reach environments like on-premises, air-gapped, or intermittently connected systems, which often house sensitive data.
You should develop strategies to ship fixes rapidly, ideally without human intervention. By streamlining the promotion and deployment process, you can reduce the time it takes to remediate vulnerabilities and enhance the overall security of your software ecosystem.
Apollo supports this with first-class automatic roll-off for reported vulnerabilities through automatic recalls based on vulnerabilities and recall roll-off strategies.
Knowing where vulnerable software is deployed allows you to assess the impact and prioritize remediation efforts. Even if you can typically roll off of problem releases without any human intervention, there are going to be edge cases that require clear, centralized reports. You may not have any releases of software which are not impacted by vulnerabilities.
Apollo Environment and Product workflows provide clear access to which versions of software are deployed from multiple perspectives. If there are problems with a release, they will be surfaced through software recall notices in-context or in aggregate statuses.
Apollo also supports a global Risk Management application and in-context Security tabs which provide a vulnerability-centric point of view. You can use these views to focus on specific vulnerabilities, or only on the ones which pose the most risk or impact the most deployed software. Each vulnerability will link out to every deployed instance which is affected by it. From these views, you can choose to take actions such as suppressing the vulnerability.