I’ve worked as a threat hunter in several Black Hat Security Conference Network Operations Centers (NOCs) across the globe. So I didn’t expect to be surprised by much when signing on to be a part of the NOC for SCinet—a conference that has the “fastest conference network in the world.” And yet I was surprised by just how diverse the SCinet NOC team was, how collaborative the environment was, and how much we were able to achieve with automation in such a short amount of time. I learned quite a bit from the experience, particularly about the importance of experimentation in security, so I wanted to share a few takeaways from my time in the SCinet NOC that I hope you can learn from too!
I am a curious person who has learned a lot of lessons from trying things and failing. Lots of things I do are experiments–some intentional, some unintentional. SCinet was also an experiment for me in the sense that it was a NOC I had never participated in before. Ultimately, I was experimenting with two questions: Could I succeed in this new conference environment? And how much of my skills from the Black Hat NOC (and prior industry experience) would translate to this network?
One surprise I didn’t anticipate is that SCinet is itself an experiment in how conference partners and operators can build a high-performance, and yet temporary, conference network. Each time the SCinet network is built, the setup changes; not just because the conference venue changes, but also because the people involved in building the network ask questions like:
“How can we improve this?”
“How can we make it faster and more resilient?”
“How can we take advantage of newer technologies and practices?”
To monitor this particular network setup, we used four Corelight AP 5000 Series sensors and Corelight’s Open NDR Platform (in particular, its Investigator dashboard), and were prepared for 400 Gbps of throughput. This was orders of magnitude more traffic than I was used to working with at other conferences.
A quote from the SCinet website sums it up nicely: “SCinet not only provides internet connectivity for conference attendees, it pushes the boundaries of networking technologies and innovations through workshops, demos, and collaborative opportunities.”
I think SecOps and NetSec teams everywhere can benefit from this experimental mindset. What ambitions can you and your team have to improve your own operations? Try something new! Even if it doesn’t go as planned, you will still learn something you can take with you into your next project.
It’s one thing to say you offer “collaborative opportunities,” and it’s another to put your packets where your port is. SCinet does this. Time and time again throughout the week of being on-site to keep the network operational, I met and had conversations with people who were taking advantage of educational opportunities offered by SCinet. These included: students without practical tech industry experience who came to get hands-on at SCinet; people who worked in one tech niche interning on a team they had no experience with just to try it out; and a programmer who had never worked in security and jumped head-first into the pool to work incidents alongside me on the NetSec team in the NOC that week.
It’s great to see that kind of investment in the future of technologists. There is a lot of talk on social media about how companies want to attract talented experts, but sometimes they don’t show any interest in doing the work to create tomorrow’s experts. Offering growth opportunities like internships and cross-functional training is exactly how every organization can contribute to creating tomorrow’s experts.
Hands-on educational experiences like SCinet have the potential to create future experts. Some people are just so talented and driven that they can make themselves an expert through sheer determination, self-education, and the right hands-on experience. Companies would have more access to expertise if they took more chances on training “unqualified” individuals who had an interest in learning. This is something I feel strongly about because, at one point in time, I wasn’t qualified either. I earned my qualifications through experience, and some of my earliest experience came from internships. In short, I say let’s give the next generation a hand.
Defending the SCinet network was fun and challenging, and rather critical work. The address space in use was on loan from another organization, so we had a responsibility to protect the reputation of that network space in the midst of thousands of people and devices being on the network. Vigilance is key in those circumstances, and I was thankful that we had a team of NetSec operators focused specifically on taking actionable Suricata IDS alerts and automating response actions from them so that code could enforce many of the rules 24/7 for us. This allowed the rest of our team to focus on other more important things, such as triaging alerts that required more context and investigation.
Automated response is a dream for many organizations, and I think that it is a dream that is within reach for many. While it may seem unattainable at first, from what I have experienced, it really can be as simple as just making automation a priority. There are lots of low-code/no-code systems around that can do a lot of things that were promised by the SOAR product segment, and there’s absolutely no shame in repurposing one of them to be your “starter SOAR.” There’s so much to learn from automating at least some of your response capabilities, and it will free up time for team members to get more creative so they can move on to more interesting threat hunts.
When getting started, it’s important to remember that just because you start to think about automating some responses to some conditions, it does not mean that you must commit to automating all responses to all conditions. Start slow, and find the balance. Automate one simple task that requires the least of your specific skillset or brain power. Once that is done, look for another thing you can automate. Over time, the efforts will add up.
All in all, I had a great, challenging, educational, and rewarding time working in the SCinet NOC for Supercomputing 2025 (SC25). I look forward to seeing what the network that takes “a year to design, a month to build, a week to operate, and one day to tear down” will look like next year when SC26 comes to my stomping grounds… Chicago! Am I going to take some people for selfies at “the bean” (Cloud Gate)? Of course. Am I going to try to make them eat deep dish pizza? Also yes. Am I going to bring some Do-Rite Donuts into the NOC? Yup. Am I going to have a blast sifting through network alerts and telemetry? It would be hard not to.
If you liked this boots-on-the-ground conference recap, be sure to check out my other blog posts about what I learned while working in the Black Hat NOC. And if you’re looking to start your adventure of diving into more interesting and advanced threat hunts, I recommend reading our Threat Hunting Guide.