Why AI threat detection is reshaping the SOC

Anomaly detection is perhaps the most well-known type of threat detection using AI. But anomaly detection is just one possible type of detection available under the umbrella of AI threat detection. Typically, anomaly detection uses unsupervised machine learning to look for deviations from a defined baseline (what is considered normal) of system, network, or entity behavior. Other types of AI threat detection can use deep learning, natural language processing, supervised or unsupervised machine learning. For example, behavior detection can use labeled data (data that has been identified), to train an AI detection engine to learn patterns and behavior around a specific type of attack, and to look for matching behavior in the environment for signs of an attack, even when other elements of a possible attack have been modified (like originating source, actual malware used, etc.)

In order to get the best results from AI threat detection, it’s essential to have diverse, high-quality security data sources. Poor data can skew results and cause false positives. It’s also important to have complete data. For example, when using AI threat detection for anomaly detection, it is important to observe an environment where activity is typical for an extended period of time. If for example, there are events that only occur monthly on a network, it’s important that the training period for the AI covers a period of time greater than a month to capture the monthly activity, and it’s important for the object (or objects) collecting the data to be situated in a location on the network capable of observing all of the relevant activity. 

Likewise, if supervised machine learning (where labeled datasets are used for training) is used for AI threat detection, the labeled data should reflect high-quality, vetted, and properly categorized data. Any miscategorized data can increase false positives and missed detections.

Sources of data that can be used for AI threat detection can include:

• Network and traffic data
• Endpoint and host data
• User and identity data
• Threat intelligence feeds
• Application and cloud data
• Security tools and alert data

In order to train an effective AI threat detection model, high-quality data sources from comprehensive data sources are required, including accurately classified examples in the case of training on labeled data.

AI threat detection can help to reduce overall false positives, especially in a multi-layered threat detection architecture.

Traditional detection can generate false positives since signature-based detection will alert on any match, even if it is benign. Threshold-based detection can also flag harmless behavior (for example, multiple accidental login failures). Even the use of AI threat detection can generate false positives, when the quality of the data being fed to AI is poor or biased. These false positives can overwhelm a SOC with additional noise and workload.

sAI threat detection using quality data can help reduce false positives through

• Contextual analysis: AI can be used to detect threats and then correlate a threat across multiple data sources and detection engines, increasing the confidence that the detection is a valid threat.
• Behavioral baselining: A fundamental basis of anomaly detection is using AI to learn what normal traffic and activity look like during a baseline period, with quality data fed during the baseline, helps reduce alerts for infrequent but harmless behavior.
• Adaptive learning: AI can update and learn as the environment evolves, unlike static detection engines. This reduces false positives caused by changes to workflows and other benign changes in the environment.
• Threat prioritization: AI can help assign risk scores to alerts and events, enabling SOC analysts to prioritize higher-scoring alerts.

AI threat detection using the best data will not eliminate all false positives, but it can significantly reduce the volume of meaningless alerts, enabling SOC analysts to focus on real threats.

To measure true AI threat detection performance, specific KPIs that capture both technical accuracy and operational impact are required. These are some of the KPIs that can be tracked to measure performance. 

Core detection accuracy KPIs

• True positive rate (detection rate/recall): % of actual threats correctly identified by AI (higher % shows better performance)
• False positive rate: % of benign events incorrectly detected as threats (lower % results in less analyst fatigue)
• False negative rate (miss rate): % of actual threats the AI missed, useful for measuring blind spots 
• Precision (positive predictive value): % of truly malicious threats detected
• F1 score: a machine learning evaluation metric that combines precision and recall scores to measure a model's accuracy, used to balance detection accuracy vs. false positives

Operational effectiveness KPIs

• Mean time to detect (MTTD): how quickly the AI detects an incident
• Mean time to respond (MTTR): how quickly analysts can respond after AI raises an incident
• Alert volume per day: total alerts generated (typically segmented by severity), to help track whether AI is flooding the SOC with noise
• Alert-to-incident ratio: how many AI alerts turn into validated incidents to determine AI efficiency 
• Automation coverage: % of incidents where AI detection with an automated response was sufficient without human intervention

Business and risk KPIs

• Reduction in successful breaches: compare incidents pre- and post-AI deployment
• Cost per incident: average cost of detection and response pre- and post-AI deployment, and continuing
• Regulatory / compliance alignment: % of alerts where explainability and audit trails are sufficient for compliance needs
• Model drift/degradation rate: how quickly detection accuracy deteriorates over time without retraining
• Analyst productivity: alerts handled per analyst per shift before and after AI implementation

In practice, SOC and security teams often track detection accuracy metrics for model health, efficiency metrics for operational impact, and business risk metrics for leadership reporting.

In an ideal environment, AI models would be continuously retrained as new data enters. But in practice, some organizations cannot tolerate the associated risk of continuous retraining or have a closed environment where continuous retraining is not possible.

In practice, organizations choose one of three retraining cadences:

• Continuous/online learning
  • For highly dynamic environments, useful for fast-evolving malware and phishing campaigns.
• Regularly scheduled retraining
  • The SOC/security team chooses the frequency of retraining, 1-3 months if there’s a strong data pipeline, 6-12 months in stable environments. Regular retraining ensures the model adapts to new attack tactics and infrastructure changes.
• Event-triggered retraining
  • Retrain after a major environment change, a change in false positive or false negative rate, or detection failures.

Any indication of model drift (drop in precision, increase in false positives and negatives, etc.) is a good indicator of a need to retrain. In addition, best practices include automated monitoring and tracking of KPIs, keeping a human in the loop, rolling A/B training (where multiple models are A/B tested before deployment), and a hybrid approach to retraining (regularly scheduled combined with event-triggered retraining).

Deploying AI threat detection in the cloud brings some great advantages, but also introduces some specific challenges. These challenges include:

• Data security and privacy
  • With the cloud, there is always concern over sensitive data and privacy. In addition, there may be concerns around data residency and compliance with regulations like GDPR and HIPAA with regard to where data can be stored and processed.
  • In addition to increased use of encryption, it may be difficult for AI to analyze encrypted traffic in the cloud.
• Multi-cloud and hybrid environments
  • With multi-cloud and hybrid environments, data may be incomplete, or data between sources may be inconsistent.
  • Standardization issues between different providers may make it difficult to integrate and fully process data.
  • Lateral movement between clouds and between cloud and on-prem may be more difficult to observe, making detection harder.
• Model performance and reliability
  • High data volume can lead to issues with scaling to accommodate the incoming data to the AI models.
  • Model drift may be more prevalent in the cloud, where workloads are constantly changing, leading to the possibility of more false positives and missed threats (false negatives).
• Integration and deployment complexity
  • Integrations between cloud, hybrid, and on-prem systems, including SIEM/SOAR and other platforms that work with AI detections, may be non-trivial.
  • API limitations with cloud services, as not all logs are exposed in real-time, which may create blind spots
  • Latency between attack and AI detection in a highly elastic cloud environment 
• Skills and operational gaps
  • Expertise shortages may exist, as many SOC teams lack deep cloud security and AI/ML expertise.
  • Explainability may be an issue with black-box AI detections, which issue alerts, but not the explanation and the evidence behind why an alert was triggered.
• Cost and resource management
  • Compute costs may be high from training AI models on large-scale cloud data
  • Data ingestion costs if there’s a need to move telemetry between regions and/or clouds
  • Operational overhead is ongoing from retraining, tuning, and monitoring in the cloud

Adversarial attacks can deliberately manipulate inputs to AI in an attempt to evade detection. There are a number of ways they attempt to evade detection. These include:

• Evasion attacks (inference-time attacks), where attackers attempt to trick AI into misclassifying malicious activity as benign using:
  • Perturbation, making tiny, carefully chosen changes to data (changing a few bytes to malware, altering traffic patterns, etc.) to make traffic look normal, while still enacting an attack.
  • Feature manipulation, by adding noise or disguising attack features, for example, padding malware with benign code.
  • Mimicking legitimate behavior by imitating a normal user or system activity to avoid detection. Attackers can use legitimate tools on the network to enact their attack, referred to as living-off-the-land (LOTL) attacks.
• Poisoning attacks (training-time attacks) are designed to corrupt the AI model during training so that the AI incorrectly detects behavior and fails to detect the attack. There are a number of ways to enact a poisoning attack. These include:
  • Label flipping, by inserting malicious samples labeled as benign and benign samples labeled as malicious.
  • Backdoor attacks involve training the model to behave normally except when it sees a specific trigger (possibly a byte sequence), at which point it misclassifies the input.
  • Data flooding occurs when the training data is flooded with misleading patterns. This can make the model less accurate overall.
• Model extraction and reverse engineering, with the goal of learning the AI model’s boundaries in order to craft an attack that is just on the safe side of the boundary. It uses techniques like:
  • Query-based probing, where it sends test inputs to see how the AI responds and then infers the logic used.
  • Surrogate models, 
• Obfuscation and adversarial examples in the wild
  • Polymorphic malware is malware where the code and structure automatically mutates every time it runs, producing endless variants.
  • Adversarial perturbations in network traffic are adding junk traffic or changing packet timing so AI classifiers miss any malicious content.
  • AI-generated evasion is where attackers use their own generative AI to test and produce attacks that bypass detection.

While AI threat detection can greatly enhance the efficiency of a human SOC analyst, it does not and typically cannot replace a human analyst. Each contributes to better threat detection.

What AI does well

• Scale and speed: AI can process terabytes of logs, network traffic, and endpoint events quickly.
• Pattern recognition: AI can detect subtle anomalies and correlations across large datasets.
• Noise reduction: AI is useful for filtering out repetitive low-level alerts and prioritizing high-risk events.
• Automation: AI can handle routing tasks, like triaging common alerts, sandboxing suspicious files, and flagging known IOCs.

What humans do better

• Context and judgement: A human analyst can understand business context, for example, identifying an alert as legitimate traffic based on business needs.
• Threat hunting: A human analyst can proactively search for novel attack techniques that AI has not seen.
• Creative reasoning: Attackers often innovate in unexpected ways, humans can adapt and hypothesize beyond what AI models were trained for.
• Incident response: AI raises alerts and can automate parts of the triage process, simplify understanding of the threat, and reduce response time, but until AI is proven reliable for triage, humans are still better at determining final containment strategy, communication, and remediation.
• Ethics and compliance: Humans are still necessary to ensure AI decisions align with policies, regulations, and legal constraints.

AI amplifies efficiency by handling repetitive and large-scale analysis. Humans add expertise and intuition to validate, investigate, and respond to attacks. Right now, AI is best used to assist detection, initial triage, prioritization, and recommendations for response as a complement to a human SOC analyst.

Corelight Open NDR gives you unmatched network visibility and precision-crafted detections that catch what EDR misses. Backed by AI and automation, you move from alert to action—faster, using the best network evidence. Elite defenders in the SOC recognize that security alerts can—and will—be missed. They know that an evidence-first strategy is their best opportunity to catch advanced adversaries in the act.

• Because risk thrives in uncertainty, the best defense is evidence.Cyber risk is an inevitable part of any organization's security posture. Uncertainty makes this risk even harder to deal with. That's why the most sophisticated defenders adopt an evidence-based approach to network security. This strategy removes uncertainty so they can make the right decision in critical moments—when an alert comes in, when a major attack is detected, or when they're remediating a breach—so they can deal with risk from the most informed position.
• Complete visibility. Gain a commanding view of your organization and all devices that log onto your network—with access to details such as DNS responses, file hashes, SSL certificate details, and user-agent strings—rapidly, without relying on other teams to respond to data requests.
• Next level analytics. Machine learning—fueled with network evidence—delivers powerful insights so you can focus on the most critical detections. Corelight’s high-fidelity, correlated telemetry powers analytics, machine learning tools, and SOAR playbooks, improving efficiency and unlocking new capabilities so that you can make better decisions—faster.
• Faster investigation. Correlate alerts, evidence, and packets tounderstand network activity and integrate that context directly into your existing workflows. Reduce false positives and your alert backlog—with no redesign or retraining necessary. You get a full view of every incident so you can validate containment and remediation.
• Expert hunting. Rich, organized, and security-specific evidence enables you to spot vulnerabilities, intruder artifacts, critical misconfigurations, signs of compromise, and undetected attacks, further mitigating risk.