File

Tenzir can read from and write to files. This includes non-regular files such as Unix domain sockets, standard input, standard output, and standard error.

When ~ is the first character in the file path, the operator substitutes it with the $HOME environment variable.

Use from_file to read files with glob patterns, automatic format detection, and file monitoring. For writing, use to_file with a print operator.

Examples

Read a file

Read from a file and parse it in the format implied by the file extension:

from_file "/tmp/file.json"

The operator automatically decompresses the file when the suffix list contains a supported compression algorithm:

from_file "/tmp/file.json.gz"

Read multiple files with a glob pattern

from_file "/var/log/*.json"

Watch a directory for new files

Use watch=true to continuously monitor for new files:

from_file "/var/log/app/*.json", watch=true

Write a file

Write to a file in a specific format:

version
to_file "/tmp/tenzir-version.json" {
  write_json
}

With compression:

version
to_file "/tmp/tenzir-version.json.bz2" {
  write_json
  compress_bz2
}

Append to a file

In case the file exists and you do not want to overwrite it, pass append=true as option:

from {x: 42}
to_file "/tmp/event.csv", append=true {
  write_csv
}

Read/write a Unix domain socket

Pass uds=true to signal that the file is a Unix domain socket:

to_file "/tmp/socket", uds=true {
  write_ndjson
}

When reading from a Unix domain socket, use from_file with a parsing pipeline:

from_file "/tmp/socket" {
  read_ndjson
}