I have been working in the cyber security space for over 25 years. I have spent time in security operations centers (SOCs) within the US Department of Defense, taught cyber warfare operators, secured large enterprise networks and, most recently, immersed myself in the world of network detection and response here at Corelight.
So a recent task seemed simple: Use a highly capable set of network detection tools to keep the Black Hat Asia 2025 conference and its attendees safe for four days. Four days? Versus 25 years of defense and problem solving? A simple task, surely.
Well, as I will explain, it was anything but simple.
Start with the jet lag: We showed up on-site after 26 hours on three different airplanes that deposited the Corelight team half-way around the world in Singapore. The adrenaline was high but sleep was low. Fortunately, I was in great hands; I would be working alongside several Black Hat NOC veterans.
The first few days were dedicated to setup and learning the architecture, which allowed me time to dip my toes into the data pool slowly. Well and good, but when Day 1 of the conference hit, that data pool became a lake within 30 minutes. The first thing that became apparent was digging through network-based security alerts and hunting for needles in stacks of needles was going to be a bit harder than I remembered or expected.
The first challenge was filtering through a new capability within the Corelight sensor that deploys YARA. If you’re not familiar, YARA is an analysis tool that can scan files in a secure sandbox using a set of signatures to look for potentially malicious files being sent over various means. Using both a generic set of YARA rules as well as some from Corelight’s partner, CrowdStrike, the notices about findings began to pour in.
Upon closer inspection, these potentially serious malware and C2 file detections were actually firing due to something much more benign. I quickly became aware that many endpoint defense products, such as Microsoft Defender and Avast Antivirus, transfer their signatures over unencrypted HTTP communication paths. This is done so that caching solutions can help distribute the network load to the millions of hosts that need these vital signatures.
Since this traffic and the signature files were unencrypted and contained the strings within their definitions that the YARA signatures were looking for, we quickly saw that we needed some way to filter YARA notices from these “trusted” content delivery networks so we could focus on potentially real file-based threats. A NOC colleague and Black Hat veteran, Ben Reardon, made quick work of the task to remove these hosts from the list of notices, which chopped the list down to a few dozen. This was the first of many insights into how the power of Corelight products can require a bit of care and feeding, depending on the environment they’re deployed within. It was a good reminder that a rule-based detection method can be heavily influenced by the signature set you use (the set we chose for the conference erred on the side of sensitivity), and that there are often exceptions to the security rules, e.g., “you should never see this byte sequence in a file in your network traffic.”
In my past work lives I’d had experience in threat hunting as well as heavy network security defense, but it didn’t really prepare me for the task at hand. So much on the networks had changed since my last set of experiences over 10 years ago. Not being the slightest bit deterred, I went to work looking for easy targets, such as authentication in the clear, attack tools and keywords, and unencrypted protocols.
Diving into the unencrypted protocols a bit more, I noticed there were some communications over Telnet. Thinking that NOBODY should be using Telnet at a conference such as this, I dug in and noticed it was coming from one of the classroom networks. I checked the class syllabus for the use of Telnet to rule out an “authorized” training use, but found nothing. Thanks to the detailed logging within the Corelight telnet log, I was able to pull out the entire telnet session and easily reconstruct it.
After looking at the data a bit closer, it appeared to contain some sort of ASCII art. “Cool, this is going to be good,” I thought. I turned to my trusty ChatGPT secure instance, which quickly reconstructed the ASCII art and even translated the language to English for me.
I was surprised by the result: someone in one of the classes apparently was a bit bored and decided to play an interactive multiplayer text based game over Telnet.
Laughing at what I found, I followed the NOC protocol and let the other members know what I’d found. Although not a serious security issue, it was a surprising find nonetheless. What struck me about it was that looking at the network data was the only reliable way to find these types of potential issues.
As the days went by I was able to find a rhythm of checking a set of pre-defined dashboards, alert and detection details from Corelight Investigator and 6-7 data queries I had crafted along the way to look for suspicious network activity.
One of the biggest takeaways was how data on the network can be both powerful and overwhelming at the same time. I recognized Corelight’s responsibility as a leading NDR vendor to build tools that make it easier for our customers to leverage the power of NDR data without getting swamped.
In the NDR space, we say the network is the source of truth, which I found to be absolutely true during these four days. What comes with that source truth are indicators that look to be of importance from a security perspective, but end up being benign for various reasons.
My Black Hat experience has given me a fresh look on the products Corelight creates and how we can further help our customers differentiate real from benign events or false threats that occur over the expanse of network data we create.
Participating in the Black Hat NOC as a threat hunter is an honor and a humbling experience all in one. I’m greatly appreciative to my Corelight mentors — Mark Overholser, Ben Reardon, Eldon Koyle, and Ignacio Arnaldo — who helped me out along the amazing journey. It was also a privilege to work side by side with Black Hat technology partners — Arista, Cisco, MyRepublic, and Palo Alto Networks — to secure and monitor the network at this invaluable industry conference.