A brief history of intrusion detection systems

The IDS journey started thirty years ago when increasing enterprise network access spawned a new challenge: the need for user access and user monitoring. As day-to-day operations grew increasingly dependent upon shared use of information systems, levels of access to these systems and clear visibility into user activity was required to operate safely and securely.

Much of the initial headway on IDS was made within the U.S. Air Force. In 1980, James P. Anderson, a pioneer in information security and member of the Defense Science Board Task Force on Computer Security at the U.S. Air Force, produced “Computer Security Threat Monitoring and Surveillance,” a report that is often credited with introducing automated intrusion detection systems. Soon after this report was released, the first model was built, born out of the same methods used by anti-virus applications: rule-based systems that constantly scanned and compared network traffic against a list of known threats.

During the late 1980’s, with a growing number of shared networks, enterprise system administrators all over the world began adopting intrusion detection systems. However, IDS presented a couple problems. First, it could only alert on known issues that had been categorized as threats on a signature list; zero-day attacks could compromise a network’s security. Second, the constant scanning and updating of a signature list was cumbersome and significant resource drain.

Intrusion detection system examples

In most cases, an intrusion detection system is either a network-based intrusion detection system (NIDS), monitoring all network traffic, or a host-based intrusion detection system (HIDS), in which it deploys on a single, protected device. It can also be deployed to monitor protocols between devices and servers, application-specific protocols, cloud-environments, or some combination (known as hybrid IDS) of these capabilities.

IDS monitoring is outside of the network, and often utilizes a TAP or SPAN port to analyze traffic in real time. It looks for any activity that is anomalous or carries signatures of established attack methods. It inspects packets for any evidence of malicious action, such as network or port scans, and attempts to exploit known network vulnerabilities. Whenever any suspicious or known attack activity is detected, the IDS generates an alert or log data.

The detection methods of an intrusion detection system typically are orientated in one of two ways:

• Signature-based, in which the system searches for pattern evidence of known attack signatures. The limitation of this method is evident when we consider the reality of zero-day attacks that do not match any known signatures.
• Anomaly-based, which leverages machine-learning capabilities to create more sophisticated models of normal network traffic and behavior. This can be a more effective method for detecting novel attack methods. However, it can also generate more false positives, since the anomalous behavior it detects may be unusual, but not illegitimate.

The different behaviors of IDS, IPS and firewalls

Since intrusion detection systems are passive systems that alert security teams but take no remedial action, they are often used in conjunction with intrusion prevention systems (IPS) and firewalls.

Firewalls, whether they are hardware or software, monitor north-south traffic, but also apply a set of security rules to block malicious or unauthorized activity. They apply filters and reject any traffic that runs afoul of preconfigured rules.

IPS and IDS are sometimes talked about as being two parts of one system, but it’s helpful to distinguish their basic functions.

IDS monitors network traffic for suspicious activity and alerts when such activity is discovered. While anomaly detection and reporting are the primary functions, some intrusion detection systems are capable of taking actions when malicious activity or anomalous traffic is detected, including blocking traffic sent from suspicious Internet Protocol (IP) addresses.

An IDS can be contrasted with an IPS, which monitors network packets for potentially damaging network traffic, like an IDS, but has the primary goal of preventing threats once detected instead of just detecting and recording threats.

Like firewalls, IPS is designed to prevent security threats by identifying and blocking malicious traffic, network packets, or connections, or modifying access control rules if there is evidence of attempted unauthorized access. Unlike IDS, it’s a proactive function.

IPS and firewalls can work in conjunction with intrusion detection systems to bring the proactive mitigation capabilities IDS lacks. Like IDS, they have become more sophisticated over time. Modern IPS also leverage machine learning and behavioral analysis to combat more sophisticated and novel attacks. Next-generation firewalls are capable of packet capture and deep-packet inspection, and can integrate with advanced threat intelligence systems.

The limitations of intrusion detection systems

What an IDS does, it generally does well, but it’s important for security teams to recognize its limitations, how attackers can outmaneuver its detection capacities, and how it might make the security operations center’s (SOC) work more challenging.

Corelight’s suite of network security analytics can help your enterprise achieve defense-in-depth and reveal known and unknown threats. Learn more about Corelight’s analytics & detections.

How IDS can complement—and be complemented by—a strong security stack

Intrusion detection systems are still an important part of most enterprises’ cyber defenses. They can assure that communications are legitimate, help identify non-malicious bugs, ensure compliance, and hold the line against established malicious attack patterns. But given the rapid evolution of cyberattacks and network complexity, it’s important to use an IDS in conjunction with other tools. Firewalls, IPS, and antivirus software can all be part of the package, but more advanced solutions are combining the features of all these tools while adding more robust capabilities that can generate more granular and targeted network analysis.

In particular, network detection and response (NDR), leverages machine learning and other advanced analytics to augment IDS’s network monitoring capacities.

NDR can harden and extend IDS capabilities in several directions, including:

• Detection of novel threats. NDR has the capacity to analyze network traffic in its entirety, rather than focusing on specific, known malicious signatures. It can be seen as an extension and improvement on anomaly-based intrusion detection systems.
  
  
• Provide alert context. Security teams waste valuable time and energy chasing false positives. NDR can present analysts comprehensive context that includes which devices are trying to communicate, which ports are being used, and how much data is included in a transfer. The added context can help reduce the number of false positives by excluding more anomalous, but legitimate, traffic from SOC analysis.
  
  
• Insights into encrypted traffic. The most advanced NDR will monitor encrypted traffic by leveraging network security solutions, such as Zeek^®, to analyze its timing, size, flow direction and other characteristics. Combined with advanced analytics, security teams can give defenders means to spot attack characteristics and methods without necessitating encryption, which is often cost-prohibitive.
  
  
• Bundle and streamline capabilities. Strong NDR can include platforms and dashboards that integrate feeds from disparate security tools and simplify SOC workflows. This can give back time and resources to overworked teams by helping them prioritize tasks and respond efficiently to legitimate threats.

Invest in the right combination of IDS and more advanced tools

Intrusion detection systems remain an important part of the security stack for most enterprises. It provides a solid baseline for network behavior, regulatory compliance, reliable redundancy for other security technologies, and solid analytics that can help SOCs pinpoint operational problems and known attack patterns.

For an advanced threat detection and protection solution, intrusion detection systems are a complementary piece of a defense-in-depth approach that layers in technologies such as next-generation firewalls, IPS, and particularly NDR, which supplies the advanced analytic capacities that can keep SOCs in front of known and unknown threats, even before they gain a foothold on the network.

Corelight’s Open NDR Pplatform is deployable on premises and in SaaS formats, and transforms activity into evidence via advanced IDS and Smart PCAP solutions. Learn more about Open NDR.