Download the release on GitHub.
Features
Section titled “Features”Cloud MISP
Section titled “Cloud MISP”VAST Cloud has now a MISP plugin that enables to add a MISP instance to the cloud stack.
Make data predicate evaluation column-major
Section titled “Make data predicate evaluation column-major”Queries without acceleration from a dense index run significantly faster, e.g., initial tests show a 2x performance improvement for substring queries.
By @dominiklohmann in #2730.
PRs 2567-2614-2638-3681
Section titled “PRs 2567-2614-2638-3681”The new experimental web plugin offers a RESTful API to VAST and a bundled web user interface in Svelte.
Rebatch undersized batches when rebuilding partitions
Section titled “Rebatch undersized batches when rebuilding partitions”Rebuilding partitions now additionally rebatches the contained events to
vast.import.batch-size events per batch, which accelerates queries against
partitions that previously had undersized batches.
By @dominiklohmann in #2583.
PRs 2513-2738
Section titled “PRs 2513-2738”We now distribute VAST also as Debian Package with every new release. The Debian
package automatically installs a systemd service and creates a vast user for
the VAST process.
Add “-total” metric keys for schema-dependent metrics
Section titled “Add “-total” metric keys for schema-dependent metrics”VAST has three new metrics: catalog.num-partitions-total,
catalog.num-events-total, and ingest-total that sum up all schema-based
metrics by their respective schema-based metric counterparts.
Disable building unit tests in Dockerfile
Section titled “Disable building unit tests in Dockerfile”VAST Cloud can now expose HTTP services using Cloudflare Access.
By @dominiklohmann in #2578.
Emit metrics from the filesystem actor
Section titled “Emit metrics from the filesystem actor”VAST now emits metrics for filesystem access under the keys
posix-filesystem.{checks,writes,reads,mmaps,erases,moves}.{successful,failed,bytes}.
By @dominiklohmann in #2572.
Enable configuration of the zstd compression level for feather store
Section titled “Enable configuration of the zstd compression level for feather store”VAST has a new configuration setting, vast.zstd-compression-level, to control
the compression level of the Zstd algorithm used in both the Feather and
Parquet store backends. The default level is set by the Apache Arrow library,
and for Parquet is no longer explicitly defaulted to 9.
By @dispanser in #2623.
PRs 2574-2652
Section titled “PRs 2574-2652”VAST now ships a Docker Compose file. In particular, the Docker Compose stack now has a TheHive integration that can run VAST queries as a Cortex Analyzer.
Changes
Section titled “Changes”Move the version string into a central JSON file
Section titled “Move the version string into a central JSON file”Building VAST from source now requires CMake 3.19 or greater.
Make feather the default store-backend
Section titled “Make feather the default store-backend”The default store backend of VAST is now feather. Reading from VAST’s custom
segment-store backend is still transparently supported, but new partitions
automatically write to the Apache Feather V2 backend instead.
By @dominiklohmann in #2587.
Change default endpoint to 127.0.0.1
Section titled “Change default endpoint to 127.0.0.1”We changed the default VAST endpoint from localhost to 127.0.0.1. This
ensures the listening address is deterministic and not dependent on the
host-specific IPv4 and IPv6 resolution. For example, resolving localhost
yields a list of addresses, and if VAST fails to bind on the first (e.g., to due
to a lingering socket) it would silently go to the next. Taking name resolution
out of the equation fixes such issues. Set the option vast.endpoint to
override the default endpoint.
Load “all” plugins by default & allow “empty” values
Section titled “Load “all” plugins by default & allow “empty” values”VAST now loads all plugins by default. To revert to the old behavior,
explicitly set the vast.plugins option to have no value.
Add memory-usage to index and catalog telemetry reports
Section titled “Add memory-usage to index and catalog telemetry reports”VAST now emits per-component memory usage metrics under the keys
index.memory-usage and catalog.memory-usage.
Remove PyVAST in favor of new Python bindings
Section titled “Remove PyVAST in favor of new Python bindings”We removed PyVAST from the code base in favor of the new Python bindings. PyVAST continues to work as a thin wrapper around the VAST binary, but will no longer be released alongside VAST.
By @dominiklohmann in #2674.
Rename vast dump to vast show
Section titled “Rename vast dump to vast show”The vast dump command is now called vast show.
By @dominiklohmann in #2686.
Arrow 10.0.0 support
Section titled “Arrow 10.0.0 support”Building VAST from source now requires Apache Arrow 10.0 or newer.
Bug Fixes
Section titled “Bug Fixes”Add a timeout to the UDS metric sink
Section titled “Add a timeout to the UDS metric sink”The UDS metrics sink no longer deadlocks due to suspended listeners.
Remove caf::skip usages
Section titled “Remove caf::skip usages”Rebuilding of heterogeneous partition no longer freezes the entire rebuilder on pipeline failures.
Fix a connection error message
Section titled “Fix a connection error message”The error message on connection failure now contains a correctly formatted target endpoint.
Remove the shutdown grace period
Section titled “Remove the shutdown grace period”VAST no longer attempts to hard-kill itself if the shutdown did not finish
within the configured grace period. The option vast.shutdown-grace-period no
longer exists. We recommend setting TimeoutStopSec=180 in the VAST systemd
service definition to restore the previous behavior.
By @dominiklohmann in #2568.
Don’t abort startup if individual partitions fail to load
Section titled “Don’t abort startup if individual partitions fail to load”VAST now skips unreadable partitions while starting up, instead of aborting the initialization routine.
Allow read access to user home dir in the systemd unit
Section titled “Allow read access to user home dir in the systemd unit”The systemd service no longer fails if the home directory of the vast user is
not in /var/lib/vast.
Clear failed partitions from the cache
Section titled “Clear failed partitions from the cache”VAST now ejects partitions from the LRU cache if they fail to load with an I/O error.