Download the release on GitHub.
Features
Section titled “Features”Add a files source
Section titled “Add a files source”The new files source lists file information for a given directory.
By @dominiklohmann in #4035.
Add the set operator for upserting fields
Section titled “Add the set operator for upserting fields”The new set operator upserts fields, i.e., acts like replace for existing
fields and like extend for new fields. It also supports setting the schema
name explicitly via set #schema="new-name".
The put operator now supports setting the schema name explicitly via put #schema="new-name".
By @dominiklohmann in #4057.
Introduce --replace, --separate, and --yield for contexts
Section titled “Introduce --replace, --separate, and --yield for contexts”The --replace option for the enrich operator causes the input values to be
replaced with their context instead of extending the event with the context,
resulting in a leaner output.
The --separate option makes the enrich and lookup operators handle each
field individually, duplicating the event for each relevant field, and
returning at most one context per output event.
The --yield <field> option allows for adding only a part of a context with the
enrich and lookup operators. For example, with a geoip context with a
MaxMind country database, --yield registered_country.iso_code will cause the
enrichment to only consist of the country’s ISO code.
By @dominiklohmann in #4040.
Introduce the every operator modifier
Section titled “Introduce the every operator modifier”The every <interval> operator modifier executes a source operator repeatedly.
For example, every 1h from http://foo.com/bar polls an endpoint every hour.
By @dominiklohmann in #4050.
Add SQS Connector
Section titled “Add SQS Connector”The new sqs connector makes it possible to read from and write to Amazon SQS
queues.
Add an email saver
Section titled “Add an email saver”The new email saver allows for sending pipeline data via email by connecting
to a mail server via SMTP or SMTPS.
Changes
Section titled “Changes”Introduce --replace, --separate, and --yield for contexts
Section titled “Introduce --replace, --separate, and --yield for contexts”The enrich and lookup operators now include the metadata in every context
object to accomodate the new --replace and --separate options. Previously,
the metadata was available once in the output field.
The mode field in the enrichments returned from the lookup operator is now
lookup.retro, lookup.live, or lookup.snapshot depending on the mode.
The bloom-filter context now always returns true or null for the context
instead of embedding the result in a record with a single data field.
By @dominiklohmann in #4040.
Bug Fixes
Section titled “Bug Fixes”Introduce --replace, --separate, and --yield for contexts
Section titled “Introduce --replace, --separate, and --yield for contexts”drop and select silently ignored all but the first match of the specified
type extractors and concepts. This no longer happens. For example, drop :time
drops all fields with type time from events.
Enriching a field in adjacent events in lookup and enrich with a
lookup-table context sometimes crashed when the lookup-table referred to
values of different types.
The geoip context sometimes returned incorrect values. This no longer happens.
By @dominiklohmann in #4040.
Add SQS Connector
Section titled “Add SQS Connector”Source operators that do not quit on their own only freed their resources after they had emitted an additional output, even after the pipeline had already exited. This sometimes caused errors when restarting pipelines, and in rare cases caused Tenzir nodes to hang on shutdown. This no longer happens, and the entire pipeline shuts down at once.
Fix disk metrics with custom state directories
Section titled “Fix disk metrics with custom state directories”Disk metrics now work correctly for deployments with a customized state directory.
By @dominiklohmann in #4058.
Fix from <url> with username
Section titled “Fix from <url> with username”from <url> now also works when the url specifies username and password.
Fix an off-by-one error when loading persisted contexts
Section titled “Fix an off-by-one error when loading persisted contexts”We fixed a bug that caused every second context to become unavailable after a restarting the node.
By @dominiklohmann in #4045.
Fix invalid assertion in compress operator
Section titled “Fix invalid assertion in compress operator”The compress and to operators no longer fail when compression is unable to
further reduce the size of a batch of bytes.
By @dominiklohmann in #4048.