🚀 Features
Section titled “🚀 Features”Add to_clickhouse operator
Section titled “Add to_clickhouse operator”Mar 18, 2025 · @IyeOnline · #5032
We have added a new to_clickhouse operator, which enables you to easily send
events to ClickHouse.
Introduce metrics "pipeline"
Section titled “Introduce metrics "pipeline"”Mar 18, 2025 · @dominiklohmann · #5024
metrics "pipeline" provides an easy way to view the ingress and egress of
pipelines. The new metrics show the ingress and egress of every pipeline in
windows of ten seconds.
Implement an assert_throughput operator
Section titled “Implement an assert_throughput operator”Mar 17, 2025 · @dominiklohmann · #5055
The new assert_throughput operators allows for checking whether a pipeline
meets the minimum expected throughput at any place in the pipeline.
Port load_gcs and save_gcs to TQL2
Section titled “Port load_gcs and save_gcs to TQL2”Mar 17, 2025 · @raxyte · #5054
The load_gcs and save_gcs operators are now available in TQL2 to interact
with Google Cloud Storage.
Add a global ca-certificates config option
Section titled “Add a global ca-certificates config option”We introduced common TLS settings for all operators that support TLS. The Tenzir
config now has a key cacert, which will set the CA certificate file for all
operators using it. The default for this will be chosen appropriately for the
system.
🔧 Changes
Section titled “🔧 Changes”Introduce metrics "pipeline"
Section titled “Introduce metrics "pipeline"”Mar 18, 2025 · @dominiklohmann · #5024
metrics "operator" is now deprecated. Use metrics "pipeline" instead, which
offers a pre-aggregated view of pipeline metrics. We plan to remove operator
metrics in an upcoming release, as they are too expensive in large-scale
deployments.
Add a global ca-certificates config option
Section titled “Add a global ca-certificates config option”The skip_host_verification option has been removed from the load_http,
save_email and save_http operators. Its functionality has been merged into
the skip_peer_verification option.
🐞 Bug Fixes
Section titled “🐞 Bug Fixes”Fix passing string params to {from,to}_fluent_bit
Section titled “Fix passing string params to {from,to}_fluent_bit”Mar 17, 2025 · @dominiklohmann · #5053
We fixed a regression that caused strings passed as options to the
from_fluent_bit and to_fluent_bit operators to incorrectly be surrounded by
double quotes.
to_fluent_bit incorrectly reported zero bytes being pushed to the Fluent Bit
engine as an error. This no longer happens.
Fix bug that caused read_zeek_tsv to produce invalid fields
Section titled “Fix bug that caused read_zeek_tsv to produce invalid fields”Mar 17, 2025 · @dominiklohmann · #5052
The read_zeek_tsv operator sometimes produced an invalid field with the name
\0 for types without a schema specified. This no longer happens.
Fix crash in from "path/to/file.ndjson"
Section titled “Fix crash in from "path/to/file.ndjson"”Mar 13, 2025 · @dominiklohmann · #5050
The from operator no longer incorrectly attempts to use parsers with a known
file extension that is a suffix of the actual file extension. For example, from "file.foojson" will no longer attempt to use the json parser by default,
while from "file.foo.json" and from "file.json" continue to work as
expected. This fixes an error for .ndjson files, which could previously not
decide between the json and ndjson parsers.