Time to Blow Up the SOC?

Your Security Operations Center (SOC) is barraged with so many alerts that your team may be shell shocked into believing that they are under a constant and unmanageable assault. Indeed, they are under siege – from a constant barrage of data. Alert fatigue is not just some industry buzz phrase – it’s a very real phenomenon that even the most well-resourced SOC teams find themselves facing.

A recent report found that 10 percent of SOC teams are inundated with more than 15,000 security alerts each and every day. And according to a Ponemon survey of IT security professionals, 37 percent of respondents faced more than 10,000 alerts per day and more than half of those were false positives, which can easily cost organizations thousands of wasted hour and millions of wasted dollars every year. Realistically, many “true positives” are for security events with incredibly low value, such as reconnaissance scans. Most scans don’t turn into an issue, and the ones that do often don’t correlate with any information that can be used to defend against the attack.

And, it’s pretty common to miss the signal through the noise—spending too much time on the low value stuff, and missing the actual attacks.

The model of gathering as many logs as possible and sending them off to be centrally analyzed is like trying to find needles in haystacks by gathering all the hay you can find in a 10 mile radius.  You could make a case for it around completeness and the ability to apply analytics, but in reality it turns out to be a horrible approach.   And when the approach does identify an attack, it tends to be hours or even days after the attack has taken hold.

The problem of false positives is much bigger than wasted resources. Anyone who remembers the tale of The Boy Who Cried Wolf can tell you that being desensitized to alarm bells can have devastating consequences. Consider that nearly one third of IT professionals admit to ignoring security alerts altogether because they are so inundated. In a nutshell, the landscape is changing at light speed, the current model is largely broken and a drastically new approach is in order.

So, if your SOC team is spending their day sorting through thousands of false and near-valueless alerts, while missing real attacks, it’s critical to ask a very basic-but-important question: do you really need a security operations center, or are you just wasting time and money? There are certainly practical arguments to be made in favor of the SOC and many organizations require one – or at the very least might consider their MSP options – given the current landscape.

Conventional wisdom is that if the threat of a breach is keeping your c-suite awake at night, a security operations center is probably a good idea. The intentions of those who have relied on the SOC model are certainly on target, as are the goals of their teams themselves. However, the fears keeping your c-suite awake at night are as likely caused by false alarms as they are by the media’s fever-pitched coverage of overhyped threats and breaches.

It’s certainly unrealistic to forego security operations, but it’s important to look at the root causes for failure, and ask if we can transform the model.

The primary problem is the quality of data. Today, even after all the raw data coming from around the network goes through a best-of-breed correlation and analysis engine, SOC teams will still find themselves drowning in a sea of alerts. The signal-to-noise ratio in security appliances is the main culprit and as the number of alerts increase, the problem is only going to get worse. Companies will need to find a way to move toward detection approaches with much lower noise levels – meaning they can no longer afford to rely on a shift away from appliances.

A secondary problem is improper staffing.  Every SOC in the world is chronically understaffed, and would be even if the alert volume were halved.  When the industry talks about there being over a million unfilled cybersecurity jobs, with that number burgeoning to 6 million by the end of the decade, most experts except the bulk of those jobs to be in a SOC.   It should be clear that, with our current approach to SOC operations, there will never be enough people for the job.

This is why much of your evaluation process should be automated. Burden your technology with the task of vetting alarm bells, so that your most seasoned analysts can spend their valuable time evaluating the most likely and interesting threats, while monitoring the truly critical events in real time.  Investing heavily in automation will ultimately allow SOCs to run with far fewer, much more highly skilled resources.

If you were to manage your detection at the machine-level, the problem of data overload and false alerts would largely disappear. On a machine, you have visibility into what’s happening on the file system, what’s happening in memory, what’s happening in the OS, and even what’s happening in the application (for common applications). That’s far more telemetry data to pick out signals and ignore the noise – as long as the data is used wisely.

While large enterprises may not be ready to shutter the windows of their SOCs quite yet, it’s important to take the most proactive approach to security alerts possible to maximize whatever resources are available to those teams and your organization. That means that neither your IT team nor your SOC can afford to waste time and effort pouring over alerts to determine which are real and which are not. In short, if something triggers an alarm, shoot first and ask questions later. This approach will help to eliminate the waste of countless hours investigating false alarms.