PCAP

PCAP Logo

PCAP is the de-facto standard format for network packet captures. VAST supports reading and writing PCAP traces via libpcap.

Import

VAST supports ingesting PCAP data either from a trace file or in “live mode” from a network interface.

File

To ingest a PCAP file input.trace, just pass it to the pcap command on standard input:

vast import pcap < input.trace

Real-World Traffic Replay

When VAST reads PCAP data from a trace, it processes one packet after next. Consequently, it puts the system and maximum load. This behavior does not work for realistic traffic replay or debugging. Thus, VAST supports a pseudo-realtime mode, which works by introducing inter-packet delays according to the difference of packet timestamps.

The option --pseudo-realtime/-p takes a positive integer c to delay packets by a factor of 1/c. For example, if the first packet arrives at time t0 and the next packet at time t1, then VAST would sleep time (t1 - t0)/c before releasing the second packet. Intuitively, the larger c gets, the faster the replay takes place.

Interface

You can also acquire packets by listening to an interface:

vast import pcap -i eth0

Flow Cutoff

Storing high-voume network traffic can quickly become prohibitive due to storage capacity constraints. Naive approaches, such as sampling or snapshot length configuration (tcpdump -s), render transport-level analysis impractical due to an incomplete byte stream.

The key strategy for efficiently recording the contents comes from exploiting the heavy-tailed nature of network traffic: most connections are short but the few large connections (the heavy tail) account for the majority of total volume. The Time Machine pioneered this appraoch by recording only the first N bytes of a connection (the cutoff) and dropping the remaining data. This allows for recording most of the connections in their entirety while achieving a massive reduction of the volume to retain.

VAST supports a bi-directional cutoff, that is, the cutoff applies to both the originator and responder TCP streams and a flow gets evicted only when both sides have reached their cutoff value.

There exist a few parameters for tuning memory utilization in addition to --cutoff (-c), which specifies the number of bytes to retain. VAST keeps a flow table with per-connection state. The --max-flows/-m specifies an upper bound on the flow table size. After a certain amount of inactivity of a flow, the corresponding state expires. The option --max-flow-age/-a controls this timeout value. Finally, the frequency of when the flow table expires entries can be controlled via --flow-expiry/-e.

Export

Exporting a PCAP trace for a given expression renders the output as binary data that can be read with any PCAP compatible tool:

vast export pcap '6.6.6.6 && 42/tcp' | tcpdump -r - -nl