Skip to content

Unified context lookups with context::lookup operator

Section titled “Unified context lookups with context::lookup operator”

Apr 1, 2026 · @IyeOnline · #5964

The context::lookup operator enables unified matching of events against contexts by combining live and retrospective filtering in a single operation.

The operator automatically translates context updates into historical queries while simultaneously filtering all newly ingested data against any context updates.

This provides:

  • Live matching: Filter incoming events through a context with live=true
  • Retrospective matching: Apply context updates to historical data with retro=true
  • Unified operation: Use both together (default) to match all events—new and historical

Example usage:

context::lookup "feodo", field=src_ip
where @name == "suricata.flow"

Include pipeline names in diagnostics and metrics

Section titled “Include pipeline names in diagnostics and metrics”

Mar 30, 2026 · @IyeOnline, @claude · #5959

The metrics and diagnostics operators now include a pipeline_name field.

Previously, output from these operators only identified the source pipeline by its ID. Now the human-readable name is available too, making it straightforward to filter or group results by pipeline name without needing to look up IDs separately.

Please keep in mind that pipeline names are not unique.

Neo support for the python and shell operators

Section titled “Neo support for the python and shell operators”

Mar 25, 2026 · @tobim, @codex · #5948

The neo execution engine now supports the python and shell operators, so pipelines that rely on subprocess-backed transformations work on the new async execution path.

For example, these pipelines now run with the neo executor:

from {x: 1}, {x: 2}
python "self.y = self.x * 2"
from {x: 1}, {x: 2}
write_lines
shell "wc -l"
read_lines

This makes it possible to keep using existing Python-based event transformations and shell-based byte-stream processing while moving to neo.

Fix crash on Azure SSL/transport errors during read and write operations

Section titled “Fix crash on Azure SSL/transport errors during read and write operations”

Apr 8, 2026 · @claude

Bumped Apache Arrow from 23.0.0 to 23.0.1, which includes an upstream fix for unhandled Azure::Core::Http::TransportException in Arrow’s AzureFileSystem methods. Previously, transport-level errors (e.g., SSL certificate failures) could crash the node during file listing, reading, or writing. Additionally, the direct Azure SDK calls in the blob deletion code paths now catch Azure::Core::RequestFailedException (the common base of both StorageException and TransportException) instead of listing specific exception types.

Reliable export for null rows in rebuilt partitions

Section titled “Reliable export for null rows in rebuilt partitions”

Apr 7, 2026 · @tobim, @codex · #5988

The export operator no longer emits partially populated events from rebuilt partitions when a row is null at the record level. Previously, some events could appear with most fields set to null while a few values, such as event_type or interface fields, were still present.

This makes exports from rebuilt data more reliable when investigating sparse or malformed-looking events.

Fix HTTP Host header missing port for non-standard ports

Section titled “Fix HTTP Host header missing port for non-standard ports”

Mar 31, 2026

The from_http and http operators now include the port in the Host header when the URL uses a non-standard port. Previously, the port was omitted, which caused requests to fail with HTTP 403 when the server validates the Host header against the full authority, such as for pre-signed URL signature verification.

Reliable recent exports during partition flushes

Section titled “Reliable recent exports during partition flushes”

Mar 30, 2026 · @tobim, @codex

The export command no longer fails or misses recent events when a node is flushing active partitions to disk under heavy load. Recent exports now keep the in-memory partitions they depend on alive until the snapshot completes, which preserves correctness for concurrent import and export workloads.