Skip to main content
Version: Next

zeek-tsv

Reads and writes Zeek tab-separated values.

Synopsis​

Parser:

zeek-tsv

Printer:

zeek-tsv [-s|--set-separator=<string>] [-e|--empty-field=<string>]
         [-u|--unset-field=<string>]

Description​

The Zeek network security monitor comes with its own tab-separated value (TSV) format for representing logs. This format includes additional header fields with field names, type annotations, and additional metadata.

The zeek-tsv parser processes this metadata to extract a schema for the subsequent log entries. Similarly, the zeek-tsv printer (re)generates the TSV metadata based on Tenzir's internal schema. Tenzir's data model is a superset of Zeek's, so the conversion into Zeek TSV may be lossy. The Zeek types count, real, and addr map to the respective Tenzir types uint64, double, and ip.

Here's an example of a typical Zeek conn.log in TSV form:

#separator \x09
#set_separator  ,
#empty_field  (empty)
#unset_field  -
#path conn
#open 2014-05-23-18-02-04
#fields ts  uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration  …orig_bytes resp_bytes  conn_state  local_orig  missed_bytes  history orig_pkts …orig_ip_bytes  resp_pkts resp_ip_bytes tunnel_parents
#types  time  string  addr  port  addr  port  enum  string  interval  count coun…t  string  bool  count string  count count count count table[string]
1258531221.486539 Pii6cUUq1v4 192.168.1.102 68  192.168.1.1 67  udp - 0.163820  …301  300 SF  - 0 Dd  1 329 1 328 (empty)
1258531680.237254 nkCxlvNN8pi 192.168.1.103 137 192.168.1.255 137 udp dns 3.7801…25 350 0 S0  - 0 D 7 546 0 0 (empty)
1258531693.816224 9VdICMMnxQ7 192.168.1.102 137 192.168.1.255 137 udp dns 3.7486…47 350 0 S0  - 0 D 7 546 0 0 (empty)
1258531635.800933 bEgBnkI31Vf 192.168.1.103 138 192.168.1.255 138 udp - 46.72538…0  560 0 S0  - 0 D 3 644 0 0 (empty)
1258531693.825212 Ol4qkvXOksc 192.168.1.102 138 192.168.1.255 138 udp - 2.248589…  348  0 S0  - 0 D 2 404 0 0 (empty)
1258531803.872834 kmnBNBtl96d 192.168.1.104 137 192.168.1.255 137 udp dns 3.7488…93 350 0 S0  - 0 D 7 546 0 0 (empty)
1258531747.077012 CFIX6YVTFp2 192.168.1.104 138 192.168.1.255 138 udp - 59.05289…8  549 0 S0  - 0 D 3 633 0 0 (empty)
1258531924.321413 KlF6tbPUSQ1 192.168.1.103 68  192.168.1.1 67  udp - 0.044779  …303  300 SF  - 0 Dd  1 331 1 328 (empty)
1258531939.613071 tP3DM6npTdj 192.168.1.102 138 192.168.1.255 138 udp - - - - S0…  -  0 D 1 229 0 0 (empty)
1258532046.693816 Jb4jIDToo77 192.168.1.104 68  192.168.1.1 67  udp - 0.002103  …311  300 SF  - 0 Dd  1 339 1 328 (empty)
1258532143.457078 xvWLhxgUmj5 192.168.1.102 1170  192.168.1.1 53  udp dns 0.0685…11 36  215 SF  - 0 Dd  1 64  1 243 (empty)
1258532203.657268 feNcvrZfDbf 192.168.1.104 1174  192.168.1.1 53  udp dns 0.1709…62 36  215 SF  - 0 Dd  1 64  1 243 (empty)
1258532331.365294 aLsTcZJHAwa 192.168.1.1 5353  224.0.0.251 5353  udp dns 0.1003…81 273 0 S0  - 0 D 2 329 0 0 (empty)

-s|--set-separator=<string> (Printer)​

Specifies the set separator.

Defaults to \x09.

-e|--empty-field=<string> (Printer)​

Specifies the separator for empty fields.

Defaults to (empty).

-u|--unset-field=<string> (Printer)​

Specifies the separator for unset "null" fields.

Defaults to -.

-d|--disable-timestamp-tags (Printer)​

Disables the #open and #close timestamp tags.

Defaults to false.

Examples​

Read a Zeek conn.log from a file:

from file /tmp/conn.log read zeek-tsv

Read compressed Zeek TSV logs in one shot:

gunzip -c *.gz | tenzir 'read zeek-tsv | ...'

Write Zeek TSV into a directory, one file per schema:

to directory /tmp/results write zeek-tsv