🚀 Features
Section titled “🚀 Features”Implement the cache operator
Section titled “Implement the cache operator”Aug 30, 2024 · @dominiklohmann · #4515
The cache operator is a transformation that passes through events, creating an
in-memory cache of events on the first use. On subsequent uses, the operator
signals upstream operators no to start at all, and returns the cached events
immediately. The operator may also be used as a source for reading from a cache
only, or as a sink for writing to a cache only.
The /pipeline/launch operator features four new parameters cache_id,
cache_capacity,cache_ttl, and cache_max_ttl. If a cache_id is specified,
the pipeline’s implicit sink will use the cache operator under the hood. At
least one of serve_id and cache_id must be specified.
Perform individual catalog lookups in lookup
Section titled “Perform individual catalog lookups in lookup”Aug 30, 2024 · @dominiklohmann · #4535
The lookup operator is now smarter about retroactive lookups for frequently
updated contexts and avoids loading data from disk multiple times for context
updates that arrive shortly after one another.
Implement unstoppable pipelines
Section titled “Implement unstoppable pipelines”Aug 22, 2024 · @Dakostu · #4513
The new unstoppable flag allows for pipelines to run and repeat indefinitely
without the ability to stop or pause.
Remove the legacy metrics system
Section titled “Remove the legacy metrics system”Aug 21, 2024 · @dominiklohmann · #4381
The new rebuild metrics contain information about running partition rebuilds.
The ingest metrics contain information about all ingested events and their
schema. This is slightly different from the existing import metrics, which
track only events imported via the import operator, and are separate per
pipeline.
🔧 Changes
Section titled “🔧 Changes”Remove special character escaping from lines_printer
Section titled “Remove special character escaping from lines_printer”Aug 26, 2024 · @raxyte · #4520
The lines printer now does not perform any escaping and is no longer an alias to
the ssv printer. Additionally, nulls are skipped, instead of being printed
as -.
Remove the legacy metrics system
Section titled “Remove the legacy metrics system”Aug 21, 2024 · @dominiklohmann · #4381
The previously deprecated legacy metrics system configured via the
tenzir.metrics configuration section no longer exists. Use the metrics
operator instead.
lookup metrics no longer contain the snapshot field; instead, the values
show in the retro field.
PRs 4455-4549
Section titled “PRs 4455-4549”Aug 15, 2024 · @dominiklohmann · #4455
The show operator is deprecated. Use the operator <aspect> instead of show <aspect>. The information from show dependencies and show build is now
available in the version operator.
🐞 Bug Fixes
Section titled “🐞 Bug Fixes”Perform individual catalog lookups in lookup
Section titled “Perform individual catalog lookups in lookup”Aug 30, 2024 · @dominiklohmann · #4535
We fixed a bug that sometimes caused the retro.queued_events value in lookup
metrics to stop going down again.
Fix crash for heterogeneous subnet lookup tables
Section titled “Fix crash for heterogeneous subnet lookup tables”Aug 26, 2024 · @dominiklohmann · #4531
context inspect <ctx> no longer crashes for lookup table contexts with
values of multiple schemas when using subnets as keys.
Fix a crash when starting export on shutdown
Section titled “Fix a crash when starting export on shutdown”Aug 26, 2024 · @dominiklohmann · #4530
We fixed an issue where the export, metrics, or diagnostics operators
crashed the node when started while the node was shutting down or after an
unexpected filesystem error occurred. This happened frequently while using the
Tenzir Platform, which subscribes to metrics and diagnostics automatically.
Fix data parser precedence
Section titled “Fix data parser precedence”Aug 22, 2024 · @jachris · #4523
IPv6 addresses with a prefix that is a valid duration, for example 2dff:: with
the prefix 2d, now correctly parse as an IP instead of a string.
Fix regression in azure-log-analytics
Section titled “Fix regression in azure-log-analytics”Aug 19, 2024 · @dominiklohmann · #4516
We fixed a regression introduced in Tenzir v4.19.2 in the azure-log-analytics
operator that prevented it from starting correctly.