A Dozen Security Questions for DevOps after Deployment

DevOps brings operations and development teams together through the whole production lifecycle, leading to faster and more agile software development. But harder, better, faster, and stronger doesn’t always mean safer. Security is a critical part of that process and without properly integrating it into every phase, you’re putting your company, and your customers, at risk.

While we prefer the term “continuous security,” it seems that SecDevOps and DevSecOps are still being adopted by the industry at large. We’ve made the case as to why terms like this are misleading, or even inadequate, but sometimes the rose by any other name may be sweet enough. The bottom line is that integrating secure development best practices and methodologies into the entire DevOps deployment chain enables a safer, more efficient and truer digital transformation for business, no matter what you call it.

Steven Arychuk over at New Relic defines the concept of integrating security into DevOps in his blog post as “a set of best practices designed to help organizations implant secure coding deep in the heart of their DevOps development and deployment processes.”

The resources invested by integrating security best practices into DevOps aren’t the roadblock people often associate with them and make the process much more secure. But what about when this all goes live?

Once you’ve launched those tested apps or containers into production, you are assuming they are now safe. But security cannot stop there. Security should be integrated beyond just the development lifecycle and included as part of a production deployment. Are you doing that today?

Here are 12 questions we’ve put together that you need to ask once your software is in production to ensure that you’re truly integrating security into the process:

1) Who has access to the system?

You need to be sure that only people who need access are granted it, and also be able to verify that the people accessing the system are who they say they are and not victims of an account takeover.

2) What network connections are being made?

Understanding what connections exist can help ensure you’re not providing bad actors with a fast track in and out. This also gives security and operations teams the ability to understand exactly what “normal” network behavior looks like. For example, monitoring for a network connection to a known entity within typical process execution allows you to enforce whitelists. The same holds true for blacklists where you could disallow connections to known foreign and or malicious entities.

3) What format/command are users leveraging to access the system?

Some commands are indicators of malicious activity or intent and any use outside of the norm flags a potential attack or account takeover.

4) How does the application interact with the underlying OS?

Knowing what typical execution and processes a system has access to within the OS will allow enhanced decisions when trying to enforce security at the application level. For example, a system running from userland should not have the ability to interact directly with the kernel.

5) How stable is the application?

Stability is going to be important to comprehend when monitoring for unusual behavior as it relates to memory or CPU spikes. If a system is typically stable and these types of spikes are taking place, it may be an indication of malicious activity and would be important to monitor.

6) Are there certain programs that can be executed based on the standard runtime of an application?

Knowing what programs are normal or natural within a system helps provide insight when a potential compromise is taking place. This is especially true in containerized environments where the set of programs and processes are explicitly defined prior to deployment and anything outside of that should raise an alert for security investigation.

7) Can your system detect when new files are introduced to the system that were not there before (a process that becomes even easier with the presence of containers and microservices)?

A new or unknown file within a system could mean an unauthorized user was able to access the system. The new file could contain malicious code that either the application could access or, if there is a logging system scanning the host, it could pick up a file and execute potentially malicious code

8) Can you detect when privileges are attempting to be escalated?

Gaining access to resources that are normally protected, especially those of an admin, is a sure sign something is wrong. This is also typically an indication that an attacker is trying to do something more. Privilege escalation could lead to someone moving laterally within an environment and it could also lead to data exfiltration issues among other things.

9) Can you monitor mission critical applications for events in runtime?

Typically, organizations have a handful of systems that are responsible for running their mission critical systems. These systems could be revenue generating for the organization and being able to have visibility into them could lead to greater stability along with consistent predictable revenue assuming those systems are redundant and can stay up and running.

10) Can you detect when default security mechanisms are being tampered with?

Examples of security mechanisms could be something as simple as SMEP / SMAP, AppArmor, SELinux, to name a few. These default settings within most modern Linux kernels are in place to enforce ring buffer principles, read and write access, application level policies and privileges, along with resource access enforcement, and so on.

11) Are you able to retroactively investigate when a security incident takes place?

You must have access to the data and events of a security incident to find out not only what happened, but prevent it from happening again.

12) Can you automate response to security incidents / alerts?

Automating responses to malicious behavior, especially at near real-time, can help minimize or even eliminate the damages of an attack. This will also aid your security organization to understand what the attack landscape looks like and provide insight into how to prevent similar attacks in the future.

This may seem like a lot, but so is the cost of a breach ($439M if you’re Equifax). All the security controls that have been baked into the SDLC are crucial to ensuring your organization is taking the proper precautions but implementing security into production is just as important. Typically, the most critical assets to an organization are running in production, and historically, organizations have little to no insight to what is taking place in those systems.

Check out our video to see how Capsule8 uses small amounts of security-critical data to help you detect and respond in real time, allowing you to catch zero-days and other malicious activity as it takes place and bring continuous security into production.

Watch Video!