feather
Reads and writes the Feather file format, a thin wrapper around Apache Arrow's IPC wire format.
Synopsis
Parser:
feather
Printer:
feather [—compression-type=<type>] [—compression-level=<level] [—min—space-savings=<rate>]
Description
The feather
format provides both a parser and a printer for Feather files and
Apache Arrow IPC streams.
Tenzir currently assumes that all Feather files and Arrow IPC streams use metadata recognized by Tenzir. We plan to lift this restriction in the future.
--compression-type
(Printer)
Specifies an optional compression type. Supported options are zstd
for
Zstandard compression and lz4
for LZ4 Frame
compression.
compress
operator?The Feather format offers more efficient compression compared to the
compress
operator. This is because it compresses
the data column-by-column, leaving metadata that needs to be accessed frequently
uncompressed.
--compression-level
(Printer)
An optional compression level for the corresponding compression type. This option is ignored if no compression type is specified.
Defaults to the compression type's default compression level.
--min-space-savings
(Printer)
An optional minimum space savings percentage required for compression to be applied. This option is ignored if no compression is specified. The provided value must be between 0 and 1 inclusive.
Defaults to 0, i.e., always applying compression.
Space savings are calculated as 1.0 - compressed_size / uncompressed_size
.
E.g., for a minimum space savings rate of 0.1 a 100-byte body buffer will not
be compressed if its expected compressed size exceeds 90 bytes.
Examples
Read a Feather file via the from
operator:
from /tmp/data.feather --mmap read feather
Write a Zstd-compressed Feather file via to
operator:
to /tmp/suricata.feather write feather --compression-type zstd