May 18, 2022 by Gregory Bell
Editor's note: This is the third in a series of Corelight blog posts focusing on evidence-based security strategy. Catch up on all of the posts here.
American novelist F. Scott Fitzgerald famously wrote that “the test of a first-rate intelligence is the ability to hold two opposing ideas in mind at the same time, and still retain the ability to function.” All experienced security practitioners learn to master this mental trick. On the one hand, they believe efforts to prevent and detect breaches will be effective. On the other hand, they diligently prepare for the day when their efforts will fail. And on that day - when prevention and detection reach their limits - what tools do defenders reach for?
The answer, in almost every case, is high-quality evidence derived from networks, systems, and other sources - evidence that makes it possible to reconstruct the mechanism of a breach, and understand its consequences.
This blog is part of a series devoted to exploring the role of evidence in information security. Previously, Brian Dye explained the motivation for an evidence-based security strategy, and Richard Bejtlich illuminated an important use case for evidence.
Here I want to talk about the attributes of high-quality evidence. What should evidence look like, in order to be useful to defenders in the heat of battle, when the next Internet-scale security event such as SolarWinds or Log4Shell unfolds?
Few forms of security evidence have all these properties, but those that do often come from open source communities. That’s not surprising, because proprietary data formats are less likely to be accepted broadly, and less likely to evolve quickly in response to new threats and defensive practices.
The phenomenon of open-source standards for security evidence is part of a broader trend. As Microsoft’s John Lambert has written in an influential essay, “there is an open approach that is currently rippling across the infosec industry.” (Lambert goes on to explore the ecosystem of open SOC tools, highlighting the potential impact of Sigma, MITRE ATT&CK™, and Jupyter notebooks.)
In the realm of network-derived evidence, Corelight’s focus, the gold standard is Zeek. Tens of thousands of organizations worldwide have deployed this powerful network security monitoring platform. In fact, if you register for SANS training on threat hunting, you may learn that skill through the lens of Zeek data.
There are many reasons Zeek is having its moment today, but the most important is related to the quality of evidence the tool provides. Zeek has all the attributes explored above: designed from the beginning for automatic collection, it also generates neutral, detailed, and highly structured evidence that’s well understood by practitioners.
So it’s not surprising that Zeek became the foundational technology in open-source network security deployments, especially in large organizations and incident response firms. Over time, the format and structure of Zeek data have evolved to meet the changing needs of security teams. That power of constant evolution is rare, and in some sense it’s a sixth desirable feature of high-quality evidence.
At Corelight, our founders created Zeek, and our technology stack was inspired by open-source network monitoring patterns. Within that tech stack, Zeek plays the role of ‘evidence collector’. Why? When traditional techniques of detection and prevention have reached their limits, evidence comes to the rescue - if we plan for that outcome. At Corelight, it’s our mission to provide the best network evidence available.
By Greg Bell, Co-founder and Chief Strategy Officer Officer, Corelight