Tenzir Node v5.1 adds more flexible ways to access and move fields, bitwise functions, and a native integration with Google Cloud Logging.
Download the release on GitHub.
Features
Section titled “Features”Experiment with a trailing ? for field access
Section titled “Experiment with a trailing ? for field access”The .?field operator for field access with suppressed warnings is now
deprecated in favor of .field?. We added the .? operator just recently, and
it quickly gained a lot of popularity. However, suppressing warnings in
top-level fields required writing this.?field, which is a mouthful. Now, with
the trailing questionmark, this is just field? instead. Additionally, the
trailing ? operator works for index-based access, e.g., field[index]?. The
.? operator will be removed in the near future. We’re sorry for the
inconvenience.
By @dominiklohmann in #5128.
Implement the move keyword
Section titled “Implement the move keyword”The move keyword may be used in front of fields anywhere in assignments to
automatically drop fields after the assignment. For example, foo = {bar: move bar, baz: move baz} moves the top-level fields bar and baz into a new
record under the top-level field foo.
The move, drop, and unroll operators now support the ? field access
notation to suppress warnings when the accessed field does not exist or the
parent record is null. For example, drop foo? only drops the field foo if
it exists, and does not warn if it doesn’t. This also works with the newly
introduced move keyword.
By @dominiklohmann in #5127.
Add a split_at_regex option to read_lines
Section titled “Add a split_at_regex option to read_lines”We added a split_at_regex option allows for the use of regular expressions to
split events with the read_lines operator.
Implement to_google_cloud_logging
Section titled “Implement to_google_cloud_logging”We added a to_google_cloud_logging operator that can send events to Google
Cloud Logging.
Try and switch the Docker Image to debian::trixie
Section titled “Try and switch the Docker Image to debian::trixie”The to_snowflake operator is now available in the arm64 Docker image.
By @IyeOnline in #5113.
Expose Arrow’s bit-wise compute functions
Section titled “Expose Arrow’s bit-wise compute functions”We added bit-wise functions to TQL, including bit_and, bit_or, bit_xor,
bit_not, shift_left, and shift_right. These functions enable performing
bit-level operations on numeric values.
By @dominiklohmann in #5129.
Changes
Section titled “Changes”Assume UTF8 in file_contents
Section titled “Assume UTF8 in file_contents”The file_contents function now returns contents as string by default.
Non-UTF-8 files can be read by specifying the binary=true option.
Try and switch the Docker Image to debian::trixie
Section titled “Try and switch the Docker Image to debian::trixie”The tenzir/tenzir and tenzir/tenzir-node Docker images now use
debian:trixie-slim instead of debian:bookworm-slim as a base image.
By @IyeOnline in #5113.
Bug Fixes
Section titled “Bug Fixes”Use a proper subpipeline for fork
Section titled “Use a proper subpipeline for fork”Operators that interact with state in the node that is not local to the
pipeline, e.g., context::update, now properly work when used inside the nested
pipeline of the fork operator. Previously, pipelines of the form fork { context::update … } failed at runtime.
By @dominiklohmann in #5133.
Fix edge case when parsing nullable lists with type conflicts
Section titled “Fix edge case when parsing nullable lists with type conflicts”Parsing of nullable lists with type conflicts could previously lead to an error under very rare circumstances. This now works as expected.
Fix TLS options in from_http
Section titled “Fix TLS options in from_http”We fixed a bug in parsing the TLS options for the from_http operator,
preventing disabling of TLS.