PCAP is sometimes referred to as the ultimate source of truth about network activity, especially when it is unauthorized or malicious in nature (the phrase, “PCAP, or it didn’t happen” has become a mantra for many security analysts.) PCAPs contain raw code that is irrefutable evidence of activity in the network. There are myriad reasons why this evidence can benefit security teams and forensics, not least of which is determining the extent of an attacker’s intrusion. The network evidence can be critical to the process of defensible disclosure, in which organizations must notify constituents of an intrusion or breach in which the full scope of damage is not easily determined.

Packet capture analysis is a valuable function for IT teams focused on issues such as packet loss, congestion, and compliance. Captured packets also can be used to simulate network conditions for testing purposes. Security operations centers (SOC) rely on PCAP to conduct forensics, either in response to an alert or a known breach, or to hunt for suspected threats.

Given the sheer volume of data generated by a network, PCAP storage costs can become cost prohibitive, especially for security teams who may want to review weeks or months of historical data to find evidence of an intruder and subsequent activity. Most teams using traditional PCAP solutions can only afford days to weeks worth of record retention. Also, locating and retrieving the exact packets needed for a given investigation can be frustrating and time consuming, like searching for a needle in a haystack. Traditional PCAP solutions are often expensive to maintain and challenging for security teams who need fast, immediate access to evidence to conduct their investigations. PCAP tools that use filters to selectively capture and connect high value packets and integrate them natively into security team workflows offer a new way of addressing this problem.

PCAP files can take a variety of forms, which analysts can choose based on the tools at their disposal and specific use cases. Some of the most common file formats include:

• Libpcap: For use with MAC OS and Linux devices, Libpcap is an open-source C++ library, and is commonly used with packet sniffing tools.
• WinPcap: Designed for Windows environments, WinPcap includes kernel-level packet filter capacity and supports remote use cases.
• Npcap: Also used with Windows devices, and supportive of software such as Wireshark.
• PCAPng: A more recent addition to the PCAP file options, it is more extensible and has more scripting capacity.

PCAP analysis tools, such as Wireshark, tcpdump, and Windump, open the PCAP files and provide an interface in which the network data can be displayed and configured based on the IT or security team’s preferences and needs.

Wireshark and tcpdump, which are open-source, are popular tools for assessing and remediating network performance issues due to their customization capabilities, interactivity, filtering options, and cost effectiveness. Proprietary tools for deeper dives into the network are also on the market.

Note that PCAP typically is not the first option for observing network traffic or responding to an alert. Administrators or security teams often start by reviewing logs or network flows pulled from routers or servers. PCAP is an option when security analysts need more information about exactly what traffic the intrusion created.

How PCAP helps SOC teams

PCAP can benefit SOC members investigating an alert, since it can be an effective way to validate a network intrusion. By analyzing the traffic coming from a suspected source of a breach, they can use PCAPs to reconstruct the attack and formulate remediation plans. From there, rules about what traffic should be monitored and how filters can be updated can be applied.

Forensic investigators also depend on PCAP and packet sniffers to recreate security events that feature novel tactics that do not trigger alerts and have no signatures. Working in concert with other security tools, the raw network data captured in packets can help forensic analysts understand zero-day exploits and how to construct defensive strategies to prepare for new attacks.

Some limitations of PCAP

It’s important to remember that while PCAP is a useful tool for network monitoring, it has limited visibility into security vulnerabilities that are not network-based. For instance an attack that begins on an embedded system or a USB thumb drive drop attack is likely beyond the scope of PCAP, at least for the initial stages of the attack that do not cross network surfaces. Additional limitations of traditional PCAP solutions include:

• Limited look-back capabilities. A security investigation may require full packet capture in the network, but this approach often leaves analysts with only a few hours or days-worth of traffic to analyze, due to prohibitive storage costs. With average attacker dwell times measured in months not minutes, this limited lookback window kneecaps an analyst’s ability to go back in time and understand what happened.
• Excessive “chair swivel.” Full packet capture does not integrate well with modern Security Information and Event Management (SIEM) workflows, forcing analysts to toggle between, or “chair swivel,” between their SIEM and other panes of glass to locate and retrieve packets. This can lead to higher costs and time commitments that make robust forensic analysis even more complex, and costly. Corelight’s Smart PCAP solution is engineered to make its network logs compatible with external SIEMs, which provides an immediate benefit of streamlined and more effective packet inspection.

• SOCs do not always control PCAP choke points. Since PCAP is a useful tool for IT troubleshooting, network administrators often set the network PCAP capture points and filters. Without direct control, SOCs may need to request changes, resulting in time delays, or be unable to set filter guidelines.
• Packet filter setup and maintenance can be challenging. Even with choke point control, SOCs can find it challenging to determine which traffic ports and filters will capture the data streams they need. Moreover, adversaries who have gained a foothold in the network can adapt their methods to evade the areas where monitoring is active.
• Prohibitive storage costs. Long term, full packet analysis is impractical for many SOCs due to the huge increase in network throughput and transmissions speeds, and limited budgets and capacity for storage. As such, many teams must approach PCAP look-back in terms of hours or days at most. However, in many cases forensics would need to extend capture to weeks, or even months, to document all the interactions related to a network breach.

How can SOCs store the right packets for longer?

The value of PCAP to SOCs can be amplified when used in conjunction with a more comprehensive network monitoring platform. Additionally, approaches to PCAP that deploy more efficient, targeted packet capture can help SOCs greatly expand the visibility window into network activity.

Since most of the data accessible via PCAP will not be relevant to a security investigation, it is extremely useful to deploy it as part of a network monitoring platform, such as Zeek^®. Zeek’s event-processing capabilities can help SOCs separate out PCAP data that isn’t relevant to their security investigations. It generates security-relevant metadata and packages it into compact, high-fidelity transaction logs that can benefit incident response or threat hunting. It brings the additional benefit of formatting all extracted files in formats that can adapt to SIEM workflows, thereby addressing one of PCAP’s ongoing pain points.

See how Zeek and the Corelight Open NDR Platform work together to convert raw network traffic into actionable intelligence.

Newer PCAP solutions can deliver more cost-effective packet capture methods that extend retention periods and accelerate packet investigations. They implement advanced controls that help SOCs separate the packets they need via PCAP retrieval links in network evidence logs. Logs and extracted files are automatically generated and linked to databases for easy access.

The Smart PCAP functionality within Corelight’s advanced Network Detection and Response (NDR) platform can be much more highly targeted on relevant packets, since it will not capture encrypted or field-based traffic (which is covered by Corelight’s files.log and file extraction capability) Corelight’s NDR platform can also parse and log the observable characteristics of encrypted protocols (e.g. SSH, RDP and SSL) and enrich these logs with hundreds of powerful encrypted insights that reveal behaviors like large file transfers over SSH or RDP brute forcing activity. This can dramatically extend the packet lookback window by targeting the small percentage of traffic that contains unencrypted, non-filed based packets.

Smart PCAP enables security

PCAP remains an important method for network administrators and IT departments to assess and improve network performance. There is still no replacement for the raw data generated for network activity when it comes to evidence of function and activity.

While PCAP does provide security teams with a powerful threat-hunting capability, its value is restricted by the fact that networks generate far too much data, much of which is not relevant to security and forensics, while the packets that could provide evidence of a security incident are often lost due to the expense of data storage.

Corelight Smart PCAP is a more security-focused approach, which can link Zeek® logs, extracted files, and detections with the packets that are needed to conduct a targeted investigation. Solutions such as Corelight’s Smart PCAP put security teams in control of the network choke points, and extend investigation lookback windows from days to weeks, or even months, by refining the package capture process. Used in conjunction with Corelight’s NDR alerts, PCAP files can be delivered to a SIEM with one click.