Corelight Bright Ideas Blog: NDR & Threat Hunting Blog

Evaluating AI Systems for Network Detection | Corelight

Written by Ujwala Bhagavatula and Mei Lam, Corelight Labs | Aug 17, 2026, 7:26:59 PM

AI system evaluation is the process of continuously assessing AI system capabilities, limitations, and performance through quantitative and qualitative measures. Across the system lifecycle, evals provide continuous assurance: Validating system behavior before deployment and detecting drift, bias, and reliability issues in production.

Evaluation is particularly challenging in AI systems because small changes to prompts, tools, or workflows can produce unexpected behavior throughout the system. In Network Detection Response (NDR) specifically, minor adjustments to tool calls or model parameters can result in a silent drift in signal, either failing to surface critical intrusions or overwhelming security operations with noise. Continuous evaluation then becomes a core requirement for developing and operating trustworthy AI systems for network detection. This article shares how Corelight approaches that problem: The custom harnesses we built, the metrics we chose, and the lessons we learned curating security data for evaluation.

Complexity of AI system evaluation

In traditional software systems, outputs are largely predictable from inputs. Behavior can be unit tested, integration tested, and reasoned about deterministically. AI systems break these assumptions. They are non-deterministic, invoking tools, maintaining state, planning actions, making decisions, and interacting with external systems at runtime. In an AI system, behavior emerges from interactions among multiple components including the underlying evolving models and their reasoning, prompts, instructions, and system context, tool availability and API behavior, memory systems and state management, planning and reasoning loops, environmental and runtime context.

In an NDR investigation, this is a multi-step process. An alert triggers, the agent decides which evidence to pull (from logs, entity context, or historical entity activity), correlates that evidence, and produces a verdict with supporting rationale. A failure at any step, such as pulling the wrong log source, misreading a baseline, or failing to account for entity context appropriately, can silently produce a wrong verdict even when every other component behaves correctly.

Since behavior arises from these interactions rather than any single component, evaluation frameworks must cover the entire system to reveal whether the overall agent performs as intended and to catch regressions when any of these components change.

Success spans multiple dimensions

Meaningful evaluation goes beyond any single measure and should track:

  • Task completion rate
  • Autonomous tool invocation rate
  • Tool selection correctness
  • Planning quality and reasoning coherence
  • Cost efficiency and latency (time-to-detection and time-to-triage matter directly to SOC workflows, not just compute cost)
  • Safety and robustness to edge cases
  • User satisfaction and output quality

Without multi-dimensional evaluation, a change that improves one metric can silently degrade another and go undetected until it reaches production. Furthermore, AI systems evolve quickly. Teams may be simultaneously adding or modifying tools, refining prompts and instructions, upgrading models, and changing memory architecture or routing logic. An evaluation suite functions as a regression test suite for agent behavior, providing a baseline against which every change can be measured before deployment.

Lessons learned: Custom evaluation frameworks

Off-the-shelf eval frameworks are useful starting points, but they make assumptions that don't always hold for production AI systems or customer data, especially in the NDR domain. Most general-purpose frameworks assume single-turn Q&A with natural-language ground truth. Neither assumption holds for NDR. An investigation is inherently multi-step (alert → enrichment → correlation → verdict), and ground truth is structured in the form of labeled PCAPs and logs. For Corelight's custom data pipelines, internal tool calls, proprietary business logic, and product-specific scoring criteria, we found it necessary to build custom harnesses that provide a higher level of precision and control for network-based cybersecurity.

Choosing the right metrics

There are many approaches to measuring model performance, and choosing the right metrics for specific use cases matters significantly. Traditional NLP metrics like BLEU and ROUGE rely on lexical overlap and were designed in the pre-LLM era: BLEU for machine translation, ROUGE for summarization. They tend to underperform when evaluating the abstractive, paraphrased, or complex outputs that LLMs typically produce.

More semantically-aware approaches like BERTScore and Sentence-BERT improve on this by measuring meaning rather than surface similarity, but still fall short for evaluating abstract reasoning or domain-specific outputs. This gap is especially pronounced in NDR. Our outputs are technical judgments, such as malicious/benign classifications, confidence scores, entity attributions, that need to match analyst reasoning rather than phrasing similarity. At Corelight, we use a reference-based framework augmented with performance metrics that give a broader view of model behavior, including LLM-as-judge evaluation for qualitative dimensions.

LLM-as-judge considerations

LLM-as-judge approaches are powerful but require care. Key considerations include:

  • Prompt quality: The greatest driver of judge reliability after the quality of the reference dataset itself. Iterating on the judge prompt during development is essential.
  • Poorly constructed prompts: These can lead to unexpected biases and obvious inconsistencies in judgement, making the evaluation unreliable. For example, if the model is not forced to reveal chain-of-thought for scoring in compliance with a strict rubric, it can output an unreliable score first and bend the logic to justify it afterward.
  • Non-determinism: Judge outputs can vary across runs. Mitigation strategies include temperature control, majority voting, score aggregations across repeated evaluations, and structured scoring rubrics.
  • Verbosity bias: Models may favor longer responses regardless of quality.
  • Self-preference bias: Models may score outputs from the same model family higher.

A combination of deterministic and LLM-as-judge methods provides the best coverage, maintaining both interpretability and flexibility.

Model behavior and continuous improvement

With our custom Corelight harness built around data pipelines and product-specific quality criteria, we can systematically probe model behavior across multiple dimensions:

  • Distributional sensitivity: How outputs shift across input types, user intents, and query phrasings, including edge cases, domain-specific phrasing, adversarial inputs, and the long tail of real user behavior.
  • Tool call and trace behavior: Where in a multi-step agent trace the model degrades, hallucinates, or makes inconsistent decisions across similar inputs.
  • Failure mode clustering: Whether failures are random or systematic, and whether they are concentrated in a particular slice of the input distribution.
  • Regression surfaces: How behavior changes as models are swapped, prompts are updated, or upstream data pipelines are modified.
  • Scoring against private criteria: Whether outputs meet business-specific quality bars that no public benchmark was designed to capture.

Quantitative metrics (such as tool invocation rates, task completion scores, and LLM-as-judge scores) show how changes to the system improve or hurt performance. But these metrics alone don't explain why the system behaved the way it did. Qualitative metrics can bridge the gap by providing interpretation and insight into the nature and root cause of failures. This can provide crucial information in security contexts, where different failure modes carry significantly different levels of risk. Together, the quantitative and qualitative metrics turn evaluation from a pass/fail gate into a diagnostic tool, providing a grounded, reproducible picture of model behavior across the conditions our system actually faces.

How evaluation supports continuous improvement

Furthermore, evaluation runs across iterations are meaningful and actionable, and they contribute to long-term improvement:

  • Model comparisons: Swap in a new model and measure performance against the same inputs and criteria, rather than relying on third-party benchmarks that may not reflect our use case.
  • Prompt and configuration iteration: Test changes to system prompts, tool definitions, or agent scaffolding with confidence that score deltas reflect real behavioral differences, not evaluation noise.
  • Regression detection: Catch when a change that improves one slice of behavior silently degrades another before it reaches production.
  • Attribution: Isolate which components of the system are driving improvements or failures, giving engineering teams actionable signal rather than aggregate scores.
  • Consistency: Explicit, versioned evaluation criteria produce reproducible, auditable results rather than relying on ad hoc human review.
  • Scaling coverage: As the product evolves, new eval cases can be added to reflect new capabilities, edge cases found in production, or expanded input distributions.
  • A shared language for quality: Evals give product, engineering, and research teams a common reference point for what "good" looks like, reducing ambiguity across decision-making.
  • Production grounding: Continuously closing the loop between real-world behavior and eval design keeps the harness honest, surfacing genuine risk rather than optimizing for metrics that have drifted from what matters.

By leveraging our custom evaluation framework, we have systematically established behavioral baselines for various model families within the Inspector MCP product. This analytical process provides actionable signal, enabling us to identify which models perform best based on specific metrics and the underlying SIEM architecture.

Over time, the eval harness becomes an institutional record of how the system has matured, where it has improved, and where risk remains.

The challenges of security data

A golden dataset is a high-quality, hand-labeled collection of inputs and expected outputs used as ground truth for model evaluation. Cybersecurity data presents unique evaluation challenges that compound the difficulty of AI evaluation, including:

  • Data density and richness: Network traffic data is information-dense and domain-specific, requiring expert interpretation to label correctly.
  • Repeatability: Unlike static benchmarks, security data is time-sensitive and environment-dependent. Two eval runs are only meaningfully comparable if the underlying system is first properly instrumented by pinning data snapshots, controlling for pipeline versioning, and accounting for drift in threat landscapes.
  • Class imbalance: The majority of network traffic is benign, but evaluation must adequately represent the smaller, higher-stakes proportion of malicious activity.
  • Tooling complexity: An evolving stack of collection, curation, and evaluation tools creates compounding sources of variation.
  • End-to-end automation: Complete automation of the eval pipeline remains a work in progress for these same reasons.

Key principles for golden dataset curation include:

  • Subject matter expert involvement is essential: In a domain like network security, subject matter experts are required to validate accuracy and relevance, particularly for interpreting malicious traffic patterns.
  • Representativeness determines value: A golden dataset is only as useful as its resemblance to real-world inputs; gaps in coverage directly limit evaluation reliability.
  • Scale matters, but development can start small: 20–30 examples are sufficient to begin tracking iterative prompt and model improvements, 30–50 provide a reasonable representative baseline, and 1,000+ examples are advisable for conclusive analysis.

At Corelight, golden datasets are built through a manual curation process: Developing question-answer pairs and complex text summaries grounded in network traffic data, using PCAPs replayed from open-source resources such as malware-traffic-analysis.net and from carefully reconstructed networks resembling real-world systems and attacks. Human expert curation remains the gold standard in this environment. It guarantees ground truth accuracy, mitigates bias, and ensures that evaluation reflects real security logic rather than artifacts of the data pipeline.

Future directions

The current evaluation work has focused on specific use cases and agent behaviors. The goal going forward is to expand coverage systematically across the MITRE ATT&CK framework, ensuring that evaluations represent a broader and more representative distribution of the threat landscape, and that new capabilities are matched by rigorous, grounded evaluation from day one. In network detection, evaluation is not a one-time benchmark; it is an operating discipline that must evolve alongside models, tools, data, and threats.