Download the release on GitHub.
Features
Section titled “Features”Perform individual catalog lookups in lookup
Section titled “Perform individual catalog lookups in lookup”The lookup operator is now smarter about retroactive lookups for frequently
updated contexts and avoids loading data from disk multiple times for context
updates that arrive shortly after one another.
By @dominiklohmann in #4535.
Implement the cache operator
Section titled “Implement the cache operator”The cache operator is a transformation that passes through events, creating an
in-memory cache of events on the first use. On subsequent uses, the operator
signals upstream operators no to start at all, and returns the cached events
immediately. The operator may also be used as a source for reading from a cache
only, or as a sink for writing to a cache only.
The /pipeline/launch operator features four new parameters cache_id,
cache_capacity,cache_ttl, and cache_max_ttl. If a cache_id is specified,
the pipeline’s implicit sink will use the cache operator under the hood. At
least one of serve_id and cache_id must be specified.
By @dominiklohmann in #4515.
Remove the legacy metrics system
Section titled “Remove the legacy metrics system”The new rebuild metrics contain information about running partition rebuilds.
The ingest metrics contain information about all ingested events and their
schema. This is slightly different from the existing import metrics, which
track only events imported via the import operator, and are separate per
pipeline.
By @dominiklohmann in #4381.
Implement unstoppable pipelines
Section titled “Implement unstoppable pipelines”The new unstoppable flag allows for pipelines to run and repeat indefinitely
without the ability to stop or pause.
Changes
Section titled “Changes”Remove the legacy metrics system
Section titled “Remove the legacy metrics system”The previously deprecated legacy metrics system configured via the
tenzir.metrics configuration section no longer exists. Use the metrics
operator instead.
lookup metrics no longer contain the snapshot field; instead, the values
show in the retro field.
By @dominiklohmann in #4381.
Remove special character escaping from lines_printer
Section titled “Remove special character escaping from lines_printer”The lines printer now does not perform any escaping and is no longer an alias to
the ssv printer. Additionally, nulls are skipped, instead of being printed
as -.
PRs 4455-4549
Section titled “PRs 4455-4549”The show operator is deprecated. Use the operator <aspect> instead of show <aspect>. The information from show dependencies and show build is now
available in the version operator.
By @dominiklohmann in #4455.
Bug Fixes
Section titled “Bug Fixes”Fix a crash when starting export on shutdown
Section titled “Fix a crash when starting export on shutdown”We fixed an issue where the export, metrics, or diagnostics operators
crashed the node when started while the node was shutting down or after an
unexpected filesystem error occurred. This happened frequently while using the
Tenzir Platform, which subscribes to metrics and diagnostics automatically.
By @dominiklohmann in #4530.
Perform individual catalog lookups in lookup
Section titled “Perform individual catalog lookups in lookup”We fixed a bug that sometimes caused the retro.queued_events value in lookup
metrics to stop going down again.
By @dominiklohmann in #4535.
Fix data parser precedence
Section titled “Fix data parser precedence”IPv6 addresses with a prefix that is a valid duration, for example 2dff:: with
the prefix 2d, now correctly parse as an IP instead of a string.
Fix regression in azure-log-analytics
Section titled “Fix regression in azure-log-analytics”We fixed a regression introduced in Tenzir v4.19.2 in the azure-log-analytics
operator that prevented it from starting correctly.
By @dominiklohmann in #4516.
Fix crash for heterogeneous subnet lookup tables
Section titled “Fix crash for heterogeneous subnet lookup tables”context inspect <ctx> no longer crashes for lookup table contexts with
values of multiple schemas when using subnets as keys.
By @dominiklohmann in #4531.