The idea behind the SIEM (and now XDR!) technologies was to provide a single engine at the heart of the SOC, aggregating data, enabling analytics and powering workflow automation. The SIEM would act as one place to train analysts and integrate a range of complementary technologies and processes. Given the efficiency that comes from centralization, I was surprised to hear that a growing number of defenders are actually using two SIEMs. Why is that?
The trend stemmed from one key source: threat hunting programs. As elite organizations moved beyond a focus on detection to investing in resilience and data-driven security, they staffed threat hunting teams to look for attacks that bypassed their detection tools. In doing so, the type of data they needed changed: instead of event data (like alerts), they needed behavioral data (providing visibility to all of what is happening in the network, not only what was pre-judged as “good” or “bad”). They needed neutral, judgment-free insight in order to use the hypothesis-driven approach for threat hunting that has been taught at organizations like SANS since 2016.
The catch of course is that behavioral data has a lot more volume than event data … and traditional SIEMs are both architected and priced based on event data. Threat hunting requires not just storing that behavioral data but supporting rapid / complex queries or machine learning workloads as well, which stresses existing deployment architectures. The direct license cost is a well understood problem, as a volume based pricing scheme is a poor fit for threat hunting needs. As threat hunting programs mature, they often evolve to support development of in-house analytics which exaggerate these problems by further stretching the SIEM.
As a result, leading defenders have been deploying a secondary SIEM or even a proper data lake for their threat hunting and in-house analytics initiatives. One SIEM was not enough. Over the last two years those were often Elastic- or Hadoop-based, depending on the prioritization between search or analytics. More recently we have seen defenders also use either next-gen SIEMs (like Humio, especially after the acquisition by Crowdstrike) or cloud-based analytics platforms (like Snowflake, Amazon’s Athena or Google’s BigQuery, often paired with cloud native ML toolkits like Databricks or Amazon Sagemaker). The goal in either approach is much faster search and a path to custom analytics at a reasonable cost, given the higher volume of data.
So if defenders are forced to take a “data lake” or “two SIEM” strategy, we are left asking “will the XDR trend solve this problem?” Once we put the marketing label aside, most of the XDR platforms are trying to provide strong and unified analytics across the most important data sources. As of today, they are heavily focused on analytics and incident response. Will or can they evolve to become full threat hunting platforms? We hope so.
Until then, based on what we have seen with sophisticated defenders we encourage everyone to formalize your threat hunting program and ideally send one or a few key analysts to SANS for training. Use the publicly available resources like Sigma queries to accelerate your team’s work. Even with a single-SIEM approach, spending time looking for the unknown will not only help you find advanced attackers, it will improve your organization’s knowledge of their network and infrastructure across the board. As that team and effort matures, take a broad look at the available search and analytics architectures - you may find viable options that didn’t exist even a few years ago.
Editor's note: Brian also discussed this topic recently with the editor of HelpNetSecurity. Check out the Q&A interview: Are separate SIEMs for threat hunting a good idea.
By Brian Dye, CEO, Corelight
