write_feather
Transforms the input event stream to Feather byte stream.
write_feather [compression_level=int, compression_type=str, min_space_savings=double]Description
Transforms the input event stream to Feather (a thin wrapper around Apache Arrow's IPC wire format) byte stream.
compression_level = int (optional)
An optional compression level for the corresponding compression type. This option is ignored if no compression type is specified.
Defaults to the compression type's default compression level.
compression_type = str (optional)
Supported options are zstd for Zstandard compression
and lz4 for LZ4 Frame compression.
compress operator?The Feather format offers more efficient compression compared to the
compress operator. This is because it compresses
the data column-by-column, leaving metadata that needs to be accessed frequently
uncompressed.
min_space_savings = double (optional)
Minimum space savings percentage required for compression to be applied. This option is ignored if no compression is specified. The provided value must be between 0 and 1 inclusive.
Space savings are calculated as 1.0 - compressed_size / uncompressed_size.
For example, with a minimum space savings rate of 0.1, a 100-byte body buffer will not
be compressed if its expected compressed size exceeds 90 bytes.
Defaults to 0, i.e., always applying compression.
Examples
Convert a JSON stream into a Feather file
load_file "input.json"
read_json
write_feather
save_file "output.feather"