Scale is a great word, because its meaning is truly in the eye of the beholder. To an astronomer, it might mean millions of light years. To a chemist, nanometers. In the network security monitoring (NSM) world, Corelight is enabling scale in two different senses of the word: management (at enterprise scale) and data (when 30% less is a beautiful thing).
Management is the more straightforward of the two. As NSM deployments grow, and in particular as they expand beyond physical sensors to include virtual and cloud environments, the ability to administer those environments easily is critical. From its beginning, Corelight has helped our customers focus on their data … accelerating incident response, finding advanced threats, uncovering new behavioral patterns … rather than systems administration.
Now that our largest customers are approaching hundreds of deployed sensors, our mission is broadening. It has led us to develop and launch our new Corelight Fleet Manager, which allows organizations to (wait for it!) manage fleets of Corelight sensors. In doing so, we are not trying to reinvent the wheel of distributed systems management. Quite the opposite – we have taken the highest impact workflows and delivered them in a streamlined user experience. That includes grouping sensors across the environment, providing role-based access control to those groups, and then automating deployment of configuration policies to them. And yes, since you asked – of course we have an available dark mode. Across light or dark, we measure our success by how few clicks are required and whether you need to open the manual. This is a user experience designed for the administrator, not the sales demo!
Enabling data scale is more nuanced, but just as compelling. We already provide tremendous flexibility in how data is both generated and exported, including:
- Fork-and-filter architecture, so you can send some or all of your logs to different destinations in parallel (a SIEM vs. S3 for example)
- Filter language, so you can easily customize the results of any given log – just keeping the entries you need
- Custom packages, which allow you to create logs with new or unique information as you need it
These options are critical to helping organizations deliver the data they need to where they need it – which is especially important for those on a volume-based SIEM pricing model.
Even with those capabilities, we heard customers ask for more – not just capabilities, but out-of-the-box content based on real-world experience. Content that looks within the data streams themselves to screen out log entries with lower security value and maximize the ROI of their data. As a result, we have created the Data Reduction packages. They create new versions of our six most popular logs (conn, http, dns, ssl, files, and weird) that apply a cost-driven filter to the information created. For example, if a host is making repeated DNS queries hundreds of times per second, then we temporarily stop producing repeats of that log entry. If identical files and certificates are moving through the network, we stop repeating those log entries for a while as well. These targeted data reductions were developed in cooperation with incident responders at some of the world’s leading organizations, and (for many) represent an attractive trade-off between downstream SIEM cost and security data coverage. Specifically, to date we have seen a ~30% reduction in data with little loss of security insight, which is a powerful combination.
As we said at the start, scale is in the eye of the beholder. If your view of data scale is “I need it all” then we can provide it. If you are budget constrained (or not on a SIEM site license!) so you want it all in nearline storage but just the reduced version in your SIEM, we can provide that, too. To each their own, we say!
Whether you are looking at management scale or data scale, we are happy to deliver these new capabilities to you. For our current customers, thanks again for your confidence, partnership, and feedback. If you haven’t worked with Corelight (or Zeek!) yet, welcome to the movement – we are looking forward to helping you at whatever scale we can.
