Download the release on GitHub.
Features
Section titled “Features”Add show serves for debugging the serve operator
Section titled “Add show serves for debugging the serve operator”show serves displays all currently active serve IDs in the /serve API
endpoint, showing an overview of active pipelines with an on-demand API.
By @dominiklohmann in #3596.
Implement a RabbitMQ-based AMQP connector
Section titled “Implement a RabbitMQ-based AMQP connector”The new amqp connector enables interaction with an AMQP 0-9-1 exchange,
supporting working with messages as producer (saver) and consumer (loader).
Implement extended deployment options
Section titled “Implement extended deployment options”The new ttl_expires_in_ns shows the remaining time to live for a pipeline in
the pipeline manager.
Add a yara operator
Section titled “Add a yara operator”The new yara operator matches Yara rules on byte streams, producing structured
events when rules match.
Add blob type for arbitrary binary data
Section titled “Add blob type for arbitrary binary data”The new blob type can be used to represent arbitrary binary data.
Add a —live option to the export operator
Section titled “Add a —live option to the export operator”The export operator now has a --live option to continuously emit events as
they are imported instead of those that already reside in the database.
Add a velociraptor operator
Section titled “Add a velociraptor operator”The new velociraptor source supports submitting VQL queries to a Velociraptor
server. The operator communicates with the server via gRPC using a mutually
authenticated and encrypted connection with client certificates. For example,
velociraptor -q "select * from pslist()" lists processes and their running
binaries.
Update the plugins submodule pointer to include extended & serialized pipeline states
Section titled “Update the plugins submodule pointer to include extended & serialized pipeline states”The new completed pipeline state in the pipeline manager shows when a
pipeline has finished execution.
If the node with running pipelines crashes, they will be marked as failed
upon restarting.
Add events field to output of show partitions
Section titled “Add events field to output of show partitions”The output of show partitions includes a new events field that shows the
number of events kept in that partition. E.g., the pipeline show partitions | summarize events=sum(events) by schema shows the number of events per schema
stored at the node.
By @dominiklohmann in #3580.
Changes
Section titled “Changes”Implement extended deployment options
Section titled “Implement extended deployment options”The new autostart and autodelete parameters for the pipeline manager
supersede the start_when_created and restart_with_node parameters and
extend restarting and deletion possibilities for pipelines.
Add blob type for arbitrary binary data
Section titled “Add blob type for arbitrary binary data”The string type is now restricted to valid UTF-8 strings. Use blob for
arbitrary binary data.
Bug Fixes
Section titled “Bug Fixes”Detect and report incomplete object in JSON parser
Section titled “Detect and report incomplete object in JSON parser”When using read json, incomplete objects (e.g., due to truncated files) are
now reported as an error instead of silently discarded.
Fix serve exiting prematurely
Section titled “Fix serve exiting prematurely”Pipelines ending with the serve operator no longer incorrectly exit 60 seconds
after transferring all events to the /serve endpoint, but rather wait until
all events were fetched from the endpoint.
Shutting down a node immediately after starting it now no longer waits for all partitions to be loaded.
By @dominiklohmann in #3562.
Fix parsing of subnet columns in zeek-tsv
Section titled “Fix parsing of subnet columns in zeek-tsv”The zeek-tsv parser is now able to handle fields of type subnet correctly.
Check for duplicate field names in zeek_tsv_parser
Section titled “Check for duplicate field names in zeek_tsv_parser”Having duplicate field names in zeek-tsv data no longer causes a crash,
but rather errors out gracefully.
By @eliaskosunen in #3578.
Fix predicate pushdown in export and other small fixes
Section titled “Fix predicate pushdown in export and other small fixes”A regression in Tenzir v4.3 caused exports to often consider all partitions as
candidates. Pipelines of the form export | where <expr> now work as expected
again and only load relevant partitions from disk.
The long option --skip-empty for read lines now works as documented.
By @dominiklohmann in #3599.
Upgrade remaining usages of the adaptive table slice builder
Section titled “Upgrade remaining usages of the adaptive table slice builder”The csv parsed (or more generally, the xsv parser) now attempts to parse
fields in order to infer their types.