Network Security Monitoring
Corelight delivers static file analysis with YARA integration
Corelight's YARA integration helps organizations increase detection rates, improve network visibility, and reduce false positives.
A few years ago, I was part of a team responding to a high-profile security incident. After the incident was resolved, I was given a list of NDR rules to add to my firewalls. The issue was that the rules were not made for Suricata®, the IDS I was using in this position at that time, so they generated false positives. With all that extra noise, I made it my goal to eliminate that excess noise. I identified a broken rule, replayed the network traffic that tripped it, edited the rule, and then rinsed and repeated until it worked as intended. It was the normal grind. I had been doing it long enough to be annoyed, but I was still pushing forward.
They say that desperation breeds ingenuity. In detection engineering, my annoyances with the amount of time it took led me to try out a new way of tuning my detection rules: I wanted to do it with AI. In this blog, I will share a few stories of how I was able to use AI to create new IDS rules, the challenges I faced, and the lessons I learned with the goal of making it easier for you in the future.
TL;DR: AI lets people working in infosec go from finding something interesting and shipping a detection to the sensor much faster, without having to wait on a specialist. AI doesn't replace rule writing, but it can reduce hours of iteration into minutes before getting to a working draft.
The old way
When I started working in infosec, I had to learn rule writing the hard way. I worked with real data and broke a lot of things. Through trial and error, I learned what the syntax actually did, what it didn't do, and where its limits are. Then I passed those hard-learned lessons to others.
In a different example, I spent a regrettably long time bashing my head against a DCSync detection rule. The goal was to alert only on fake sync requests, but it kept throwing false positives every time a network printer enumerated the Active Directory server to figure out where to drop scanned images. Once again, I had to tune the rule the hard way. It was pure persistence. Eventually, I got the rule to a place I trusted, but it took a long time. Many of you have gone through similar experiences in your careers, but this isn’t the only way to learn.
There are formal paths to mastering rule writing: SANS and OISF both offer well-regarded training. For a stretch of my career, I ran the customer training program at Corelight and taught Suricata. By the end of a 1- or 2-day course, a strong student could read and edit rules and write basic ones from scratch. But writing rules well enough to ship in production, though, took more reps.
The expected path was much more rigid.
- You had to do some classroom time, plus whatever it took for you to step away from your day job.
- You practiced in a controlled environment, and learned the basics.
- Once training was over, you still faced a long ramp-up before you'd write something production-worthy without a second set of eyes.
Engineering detections was a specialist track, and the specialists were rare. The barrier to entry was much higher and the cycles required to master the skill were more than most had available to sacrifice. But thanks to AI, the timeline required to create your first production rule has been shortened drastically.
AI tools
In the context of rule writing, there are two categories of tools that I want to highlight that reduce the time required for detection engineering. Both are effective in reducing the traditional barriers to entry.
The first category is frontier coding harnesses like Claude Code, Codex, Gemini CLI, or any capable agent you can point directly at a file. If you already operate in a terminal with one of these wired up, you have exactly what you need to start building rules. Give your tool a bit of context or a skill, and it'll read rules, refactor yours, draft new ones from a description, and maybe even tell you where the false-positive landmines are.
The second category is purpose-built GPTs. Some of the easiest tools I’ve found to work with were created by Jason Haddix at Arcanum Information Security. He has built a set of custom GPTs aimed specifically at detection engineering. The two I use the most are the Arcanum Suricata Bot and the Arcanum YARA Bot. They're system-prompted GPTs sitting on top of the official docs and rule-writing references, which means you don't have to teach them the syntax. You just tell them what you're trying to catch. No prompt engineering required. You can find the full list of Arcanum bots at arcanum-sec.com/bots.
For full transparency, Jason and I have no professional relationship. I simply use the bots because they help take a lot of the rinse-and-repeat iterations out of the rule creation process.
When the job changed
The moment the job changed for me personally came while I was working on a YARA detection. I was trying to create a rule for a polymorphic ELF dropping a file on a Linux host. There was server-side polymorphism, per-victim recompilation, and variant packers: the whole mess. Static signatures rot while you're writing them. I was iterating the normal way when I tried the Arcanum YARA Bot on a whim. I gave it my high-level goals: I have a polymorphic ELF binary dropping a file. Here's what I am seeing. Here's what I want my alert to show. Here's what I want to ignore.
The Arcanum YARA Bot came back with three versions of the rule. Every one of them was better than what I could have written myself, even with my years of training. And it was not even close. They were just better. One version was tighter on the file-context side. Another traded off coverage for speed in a way I hadn't considered. The third was the one I ended up shipping after a round of edits, adding in some of what I learned from the other two versions.
This experience was the moment that the training-time question changed for me. It stopped being, "How many weeks does it take to make a new rule writer?" and started being, "How many minutes does it take to get a better starting draft than I could produce by myself?"
A worked example on the wire
So let’s work on a recent real-world example and let AI do some of the heavy lifting. While I was writing this post, I wasn’t able to find a Suricata rule for this case. As you read along, I invite you to walk through the steps on your own. Try writing a new Suricata rule right now! Let’s see how it turns out.
To start, pick any recent auth-bypass vulnerability. Don’t worry too much about which one to pick. The steps we go through in this section should be able to apply with little modification. In this example, I’ll use CVE-2026-41940. It is recent, it is actively being exploited, and its mechanics are public.
Next, summarize what the vulnerability is. In our case, the CVE is a cPanel/WHM auth bypass that CISA added on KEV on May 3, 2026. There is an exploit kit floating around GitHub called cPanelSniper. The bug itself is a session-file CRLF injection. When a user logs into cPanel, the server creates a session file containing line-by-line user details that look something like this on the wire:
However, the login form was not sanitizing its inputs before writing them to disk. That means that an attacker was able to send a GET / request with a crafted Authorization: Basic header whose decoded value had literal \r\n bytes in it. Then, cPanel's Session.pm would write those bytes to disk. The new lines could then become new fields in the session file (e.g., user=root, hasroot=1, tfa_verified=1). Once written to disk, the next request would come back as if the attacker were the root user.
When working with auth vulns, the evidence available on the wire (in PCAPs or logs) is usually pretty straightforward, but only if you anchor on the right thing. In the context of Suricata, the most obvious anchor, Authorization: Basic containing user=root, is actually the wrong one. By Monday, someone will have reordered the fields, swapped user=root for uid=0, or moved the injection into the cookie field. A much more durable anchor is the protocol violation underneath. HTTP header values aren't allowed to contain \r\n. RFC 7230 is explicit about it. This one fact is what defines the whole bug class, not just this CVE.
So, we dump this information into an AI tool and see what it spits out. Assuming your IDS can see decrypted traffic, a Suricata rule created by AI on that anchor might look something like this:
A hunter reading this for the first time has to chew on a few things, and an AI Agent can answer all of them in a minute. So ask it for some help! For example, v1 of our rule used http.header.raw instead of http.header, because the normalized buffer can fold or split header lines. A literal mid-value \r\n is exactly what we need to see. It chose to look at unnormalized bytes only. The PCRE is bounded {1,2048} because unbounded greedy matching on a 16KB cookie-stuffed header backtracks and eats CPU. Any reviewers worth their salt would reject the rule on sight if it weren't bounded. The http.header_names prefilter is what is doing the real work. PCRE is expensive, and most HTTP requests don't carry an Authorization header at all. Skipping the regex on those packets is what keeps the rule cheap. The addition of flow:established,to_server means that this rule won’t fire on scan traffic that never finished a handshake. Note that there is no content:"user=root" anywhere, because that's the IOC trap described above. CRLF is the behavior. The field name is just the costume that the attacker is wearing this week.
The false-positive surface here is close to zero. Real clients don't put raw newline characters in Authorization values. Ever. The thing to watch isn't false positives, it's coverage gaps. This rule won't catch a variant that injects through Cookie or X-Forwarded-For. So we need a second rule that can cover the same PCRE but with a different prefilter. It also won't help on HTTP/2, where \r\n isn't the framing delimiter or where HPACK rejects control bytes lower in the stack. Both of those are good follow-up rules, and they're also the reason behavior-class detections age better than IOC detections. The next CVE that abuses header injection might not look like cPanelSniper at all, but this rule will still catch it.
The skill required (a) to understand how each line in the rule works and what it covers and (b) to decide whether the rule is good or not takes time to develop. This is what AI cannot really replace. However, you are already miles ahead of where you would be without AI. From this point, you ask colleagues, friends, or your favorite LLM for edits or suggestions. You could scope the rule to $HTTP_SERVERS for production, add a correlated signature for the /cpsess*/json-api/version post-exploit probe, or generalize the CRLF check to other sensitive headers. You could add a fast-pattern match to minimize CPU usage or generalize the pattern to other HTTP methods. The iteration loop doesn't go away. It just starts somewhere a lot better than user=root.
A worked example on the file
Suricata rules like the one discussed above catch an exploit attempt in flight. However, it’d be nice to be able to catch the payload as well. The sensor can pull it off the wire, so we can formulate a YARA rule to look for some of our specific indicators. This is the same campaign, but we are looking at two different layers. The overlap is the point: defense in depth.
There's no malware to scan for in this case. cPanelSniper doesn't drop a webshell or a PE. Post-exploitation rides on legitimate cPanel APIs using the forged session. What it does drop is a very distinctive payload into the request body: four session fields injected via the CRLF bug. user=root, hasroot=1, tfa_verified=1, and successful_internal_auth_with_timestamp=9999999999. That last one is the giveaway. It's a max-int timestamp that no genuine cPanel login will ever produce. If the sensor extracts the request body, those fields ride along inside it.
Again, we can start by feeding a YARA-ready agent all of the information we have, similar to what we did with the bot for the Suricata example. Add any other relevant details regarding your environment and the TTPs found in the adversarial tool. Then let AI do its work. Pretty soon, you should have an output that looks something like this:
Now that we have a v1, we jump into the same feedback loop we used for the Suricata rule. Hand it to your favorite AI assistant. Ask why each string is there, why only 3 of the first 4 strings ($a, $b, $c, $d) need to match instead of all of them. Find out why the file size cap is what it is. You’ll learn key pieces of detection engineering in the process. For example, the first string is what determines the privilege level of the cPanel session. The second string is what enables an attacker to bypass MFA. The timestamp value is the campaign’s unique signature. And the fourth line is what defines the active username for the session. Choosing to match only 3 of those 4 strings gives you slack if one changes or if the request gets chunked or partially extracted. 256KB is generous on purpose because extracted HTTP bodies sometimes arrive wrapped in multipart boundaries. Take time to learn about each line, how it works, what it covers, and what it might overlook.
Next, we tune. We should drop the strict Authorization: Basic string because it's a brittle IOC that may change. Instead, let's write a second rule focused on the post-exploitation addadmin call to catch the attacker establishing persistence.
We can also push back on the AI; make it justify its logic: Why did you structure the rule this way? How can we improve it to catch future variants or entirely new exploit families? I can already think of a few more ways to push this further toward behavioral detection and away from static IOCs. Can you? See if your AI can.
Two rules on the same CVE are not redundant work. Some scenarios fit Suricata better, some fit YARA better, and many of the interesting ones want both. Suricata sees the request live and can anchor on protocol-level facts: no payload can hide \r\n in a header value, an HTTP/2 pseudo-header might be out of order, or a TLS ClientHello might have a JA3 you've never seen on that VLAN. By contrast, YARA sees the bytes after the sensor pulls them off the wire and can anchor on the payload shape that survives header rotation, encoding tricks, and field reordering. Pick the layer where the anchor is hardest for the attacker to move, and let the overlap give you corroboration when both fire on the same conn UID.
This is what detection engineering can look like when AI shortens the engineering loop.
Where these rules live on a Corelight sensor
A Corelight Sensor runs Suricata for packet-level signature work and utilizes YARA for static file analysis on files copied off the wire and exported to external storage. When a Suricata rule matches, you get both packet context and an actionable notice. Similarly, when a YARA rule matches on an extracted file, you get a notice with the file metadata already stitched to the connection that carried it. From there, these notice logs flow into Investigator, your SIEM, or whatever platform you're already using.
Running YARA directly on the sensor means analyzing file bytes exactly as they cross the wire, allowing you to catch incoming threats before the endpoint ever sees them. It sits as an additional layer of security alongside the endpoint and the SIEM, operating at a fundamentally different layer to accomplish a distinct job.
Ultimately, the rules you write describe the file by the way it arrived, not what it did after. That matters because the time from "I saw something interesting during a hunt" to "I shipped a rule that catches it going forward" can be shortened significantly. Assuming you have relevant PCAPs, you can find a threat in Investigator or your SIEM, draft the rule with an AI helper, and test it against a known-malicious PCAP or a file corpus. Once validated, you can immediately push the rule to your sensor fleet. Detection engineering used to be a different team's job. Now it can be part of the same hunt.
Words of advice
In these examples, we created detections that were assisted by AI. AI-assisted is not the same as AI-authored. That distinction is the whole ballgame. You have to have a human in the loop. You have to consult others who have more experience. And you have to test, test, test.
Frontier models can sometimes hallucinate Suricata keywords and YARA modifiers that don't exist. A draft that looks correct can fail to compile or, worse, compile but silently under-match. Over-broad regex is another common failure mode. AI tends toward generous patterns, and a generous pattern in production becomes a false-positive storm at 3:00am. A rule without a test corpus and a sample set also can't be tuned or retired responsibly. This is the craftsmanship critique Florian Roth has been pushing for a while now, and he's right about it.
Keep the human in the loop. Compile every draft. Run it against known-good traffic or a file corpus you trust. Review every false-positive and false negative. Try to identify patterns and to rectify them before you ship. If the AI helper can't explain why a specific token is in the rule, it probably doesn't belong there. And never merge an AI-generated rule into a shared ruleset (Emerging Threats, CrowdStrike, YARA-Forge, or your org's production set) without at least one other human reviewing it. AI has a lot of information, but it doesn’t have the wisdom relating to your needs and environment.
Remember that AI is not taking over the whole process. It's a shortcut to a better first draft and an accelerator for validation as you interact with the rules and the samples to tune them.
The practical takeaway
The old barriers to entry in detection engineering were all centered around time. But now, that’s no longer the blocker. You can, and should, start experimenting today. It will take time to become an expert. Formal training still works, and I recommend it to anyone who wants to go deeper. But you don’t need to wait for training to start writing rules on your own.
A single hunter who finds something interesting and can describe it in plain English is able to get a draft rule that's often better than what they could have written themselves, edit and test it, and then push it to a sensor that will start catching the pattern as soon as it crosses the wire. You probably already have the tools. All you need is a frontier coding harness like Claude Code, Codex CLI, or Gemini CLI. And if you want a free option, the Arcanum Suricata Bot or Arcanum YARA Bot are great choices.
Let AI get you to a better starting point. A Corelight Sensor can further shorten the path from hunt to production. But the human will always own the last mile.