Tenzir Node v5.2 brings our most requested feature to life, adding the ability to watch a local filesystem directory or a blob storage bucket for new files within a pipeline.
Download the release on GitHub.
Features
Section titled “Features”Add from_file operator
Section titled “Add from_file operator”The new from_file operator can be used to read multiple files from a
potentially remote filesystem using globbing expressions. It also supports
watching for new files and deletion after a file has been read.
read_lines now has an additional binary=true option which should be used if
the incoming byte stream is not valid UTF-8.
Bug Fixes
Section titled “Bug Fixes”Handle empty files in read_xsv & friends
Section titled “Handle empty files in read_xsv & friends”We fixed a bug that caused read_xsv & friends to crash when trying to read an
empty file.
By @IyeOnline in #5215.
Fix hang for operators with infinite idle timeout
Section titled “Fix hang for operators with infinite idle timeout”We fixed a hang in the cache and buffer operators when their input finished.
This also prevented the node from shutting down cleanly.
By @dominiklohmann in #5219.
Fix a crash in to_clickhouse and bump clickhouse-cpp
Section titled “Fix a crash in to_clickhouse and bump clickhouse-cpp”We fixed a bug in to_clickhouse that caused the operator to crash when
encountering lists.
By @IyeOnline in #5221.
Add from_file operator
Section titled “Add from_file operator”The read_lines operator now validates that the incoming lines are UTF-8.