Why DevOps teams need to take container security seriously [Q&A]

Earlier this year hackers were able to exploit container platform Kubernetes to install cryptomining software in Microsoft Azure.

Fei Huang, chief strategy officer at container security platform NeuVector believes that this should be a wake up call to get the attention of enterprise DevOps and DevSecOps teams. We spoke to him to find out more about the risks and how they can be addressed.

BN: How big a deal is the recent cryptomining attack, what went wrong, and what could have prevented it?

FH: Hackers infiltrated the Kubernetes system by exploiting a misconfiguration that allowed access into vulnerable containers. In a situation like this, attackers escalate their permissions and get into the Kubernetes API service or dashboard from the internal network. Malicious cryptomining containers were then deployed to multiple clusters -- all within minutes. This incident clearly demonstrates the runtime security risks that arise when Kubernetes clusters go to production, and this well-publicized attack should be a lesson for any enterprise. Make no mistake about it, Kubernetes itself is now a targeted attack surface. Network and insider attacks are major threats that can escalate extremely rapidly and must be proactively secured against.

Kubernetes-based platforms can deliver tremendous value because they are automated, distributed, and easy to scale. The rise of Kubernetes adoption among enterprises is certainly well-deserved. However, hackers can also leverage these powerful capabilities to achieve their nefarious goals -- spreading attacks faster, hiding malicious code deeper, and changing methods more frequently. These threats require a new approach to security. To specifically address these cryptomining incidents, effective security strategies should include a Layer 7 container firewall to secure ports and block suspicious traffic, behavioral lockdown capabilities that prevent containers from performing cryptomining or other abnormal activities, CI/CD pipeline vulnerability management that hardens the attack surface by reducing the number of vulnerable containers being deployed, benchmark and compliance auditing to harden the container environment's security posture, and security policy as code to enable automation.

BN: With DevOps teams racing to take advantage of containers, how much of a challenge has it been for some businesses to ensure compliance with industry regulations (like HIPAA in the healthcare industry, or PCI DSS in financial services)?

FH: In traditional technology environments, it's long been standard practice to incorporate checks that ensure security measures are aligned with regulatory compliance standards. This same practice must be applied for container-based environments and applications.

That said, container environments include new surfaces that require secure protections -- Kubernetes included. For example, PCI DSS, GDPR, and other major regulations require full visibility and control of personally identifiable information (PII) within the container environment. All containers and container communications that include this sensitive data must be carefully secured. And achieving this security in a container environment is a challenge that goes beyond the capabilities of traditional security solutions.

Another challenge: as containers scale out, the environment may span multiple regions or even multiple clouds. Security implementations that provide compliance checks and enforcement must be capable of scaling quickly with these environments. Enterprise DevOps and DevSecOps teams need to not only take care of the full build-ship-run pipeline process, but also utilize appropriate security tools to maintain evident compliance throughout the full containerized application lifecycle.

BN: What does Kubernetes security have in common with legacy network security risks?

FH: One risk that Kubernetes and legacy network security have in common is runtime attacks. Hackers will target any open ports in a network. Also, SQL injection attacks could break into applications that use MySQL -- it doesn't matter if they're containerized or not. Layer 7 application threats keep growing, and data breaches utilizing these techniques continue to make headline after headline. Kubernetes platforms add a base layer of virtualization that makes it harder for legacy network security products to detect complicated network attacks, because container platforms naturally encapsulate or encrypt some sources of truth.

BN: What's the biggest challenge enterprise DevOps and DevSecOps teams currently face with container security?

FH: I think that for DevOps and DevSecOps teams tasked with securing enterprises’ container platforms today, the biggest challenge is security automation. Manual security management isn't fully effective, and can really bog down the DevOps process. The question of how to bake in security enforcement at every stage -- from build to production -- is key to overall application growth and scalability. Container workloads are ephemeral, but security policies must be persistent. Events monitoring, analysis, and response are good practices, but cannot enforce security in-line. The good news is that today, DevSecOps security automation approaches like 'security policy as code' are available to achieve this.

BN: What steps can be taken to mitigate these security challenges for DevOps?

FH: To get started with Kubernetes security, I suggest a Kubernetes-native security strategy. You want a security implementation you can deploy, manage, and upgrade in the same way as you already manage all other container applications. The next step is building security into every stage, from build to ship to run to production. CI/CD integration, vulnerability management, registry scanning, and a container firewall are all necessary pieces to the Kubernetes security puzzle.

It's also important to check the security postures and configurations of the whole Kubernetes environment. Ensure that the environment is aligned with CIS benchmark best practices and has security features such as admission controls, role-based access control (RBACs), and so on. To prepare for production, a true layer 7 container firewall is critical to protect against runtime attacks and other issues such as those seen with this recent high-profile cryptomining hack. Finally, the last step is security automation. It’s crucial to automate all security capabilities across the pipeline, and at runtime. Enterprise DevOps and DevSecOps teams looking to strengthen and automate their Kubernetes environment security for the long haul will likely find it much more straightforward than they might otherwise think.

Photo Credit: anathomy/Shutterstock