🚀 Features
Section titled “🚀 Features”Make data predicate evaluation column-major
Section titled “Make data predicate evaluation column-major”Dec 6, 2022 · @dominiklohmann · #2730
Queries without acceleration from a dense index run significantly faster, e.g., initial tests show a 2x performance improvement for substring queries.
Add “-total” metric keys for schema-dependent metrics
Section titled “Add “-total” metric keys for schema-dependent metrics”Nov 4, 2022 · @Dakostu · #2682
VAST has three new metrics: catalog.num-partitions-total,
catalog.num-events-total, and ingest-total that sum up all schema-based
metrics by their respective schema-based metric counterparts.
Enable configuration of the zstd compression level for feather store
Section titled “Enable configuration of the zstd compression level for feather store”Nov 1, 2022 · @dispanser · #2623
VAST has a new configuration setting, vast.zstd-compression-level, to control
the compression level of the Zstd algorithm used in both the Feather and
Parquet store backends. The default level is set by the Apache Arrow library,
and for Parquet is no longer explicitly defaulted to 9.
Cloud MISP
Section titled “Cloud MISP”Oct 10, 2022 · @rdettai · #2548
VAST Cloud has now a MISP plugin that enables to add a MISP instance to the cloud stack.
Rebatch undersized batches when rebuilding partitions
Section titled “Rebatch undersized batches when rebuilding partitions”Sep 23, 2022 · @dominiklohmann · #2583
Rebuilding partitions now additionally rebatches the contained events to
vast.import.batch-size events per batch, which accelerates queries against
partitions that previously had undersized batches.
PRs 2567-2614-2638-3681
Section titled “PRs 2567-2614-2638-3681”The new experimental web plugin offers a RESTful API to VAST and a bundled web user interface in Svelte.
Disable building unit tests in Dockerfile
Section titled “Disable building unit tests in Dockerfile”Sep 14, 2022 · @dominiklohmann · #2578
VAST Cloud can now expose HTTP services using Cloudflare Access.
Emit metrics from the filesystem actor
Section titled “Emit metrics from the filesystem actor”Sep 14, 2022 · @dominiklohmann · #2572
VAST now emits metrics for filesystem access under the keys
posix-filesystem.{checks,writes,reads,mmaps,erases,moves}.{successful,failed,bytes}.
PRs 2574-2652
Section titled “PRs 2574-2652”Sep 12, 2022 · @KaanSK · #2574
VAST now ships a Docker Compose file. In particular, the Docker Compose stack now has a TheHive integration that can run VAST queries as a Cortex Analyzer.
PRs 2513-2738
Section titled “PRs 2513-2738”We now distribute VAST also as Debian Package with every new release. The Debian
package automatically installs a systemd service and creates a vast user for
the VAST process.
🔧 Changes
Section titled “🔧 Changes”Load “all” plugins by default & allow “empty” values
Section titled “Load “all” plugins by default & allow “empty” values”Nov 18, 2022 · @Dakostu · #2689
VAST now loads all plugins by default. To revert to the old behavior,
explicitly set the vast.plugins option to have no value.
Rename vast dump to vast show
Section titled “Rename vast dump to vast show”Nov 10, 2022 · @dominiklohmann · #2686
The vast dump command is now called vast show.
Arrow 10.0.0 support
Section titled “Arrow 10.0.0 support”Nov 9, 2022 · @Dakostu · #2685
Building VAST from source now requires Apache Arrow 10.0 or newer.
Remove PyVAST in favor of new Python bindings
Section titled “Remove PyVAST in favor of new Python bindings”Oct 29, 2022 · @dominiklohmann · #2674
We removed PyVAST from the code base in favor of the new Python bindings. PyVAST continues to work as a thin wrapper around the VAST binary, but will no longer be released alongside VAST.
Make feather the default store-backend
Section titled “Make feather the default store-backend”Sep 23, 2022 · @dominiklohmann · #2587
The default store backend of VAST is now feather. Reading from VAST’s custom
segment-store backend is still transparently supported, but new partitions
automatically write to the Apache Feather V2 backend instead.
Move the version string into a central JSON file
Section titled “Move the version string into a central JSON file”Building VAST from source now requires CMake 3.19 or greater.
Add memory-usage to index and catalog telemetry reports
Section titled “Add memory-usage to index and catalog telemetry reports”VAST now emits per-component memory usage metrics under the keys
index.memory-usage and catalog.memory-usage.
Change default endpoint to 127.0.0.1
Section titled “Change default endpoint to 127.0.0.1”We changed the default VAST endpoint from localhost to 127.0.0.1. This
ensures the listening address is deterministic and not dependent on the
host-specific IPv4 and IPv6 resolution. For example, resolving localhost
yields a list of addresses, and if VAST fails to bind on the first (e.g., to due
to a lingering socket) it would silently go to the next. Taking name resolution
out of the equation fixes such issues. Set the option vast.endpoint to
override the default endpoint.
🐞 Bug Fixes
Section titled “🐞 Bug Fixes”Allow read access to user home dir in the systemd unit
Section titled “Allow read access to user home dir in the systemd unit”The systemd service no longer fails if the home directory of the vast user is
not in /var/lib/vast.
Add a timeout to the UDS metric sink
Section titled “Add a timeout to the UDS metric sink”The UDS metrics sink no longer deadlocks due to suspended listeners.
Clear failed partitions from the cache
Section titled “Clear failed partitions from the cache”VAST now ejects partitions from the LRU cache if they fail to load with an I/O error.
Fix a connection error message
Section titled “Fix a connection error message”The error message on connection failure now contains a correctly formatted target endpoint.
Don’t abort startup if individual partitions fail to load
Section titled “Don’t abort startup if individual partitions fail to load”VAST now skips unreadable partitions while starting up, instead of aborting the initialization routine.
Remove the shutdown grace period
Section titled “Remove the shutdown grace period”Sep 8, 2022 · @dominiklohmann · #2568
VAST no longer attempts to hard-kill itself if the shutdown did not finish
within the configured grace period. The option vast.shutdown-grace-period no
longer exists. We recommend setting TimeoutStopSec=180 in the VAST systemd
service definition to restore the previous behavior.
Remove caf::skip usages
Section titled “Remove caf::skip usages”Rebuilding of heterogeneous partition no longer freezes the entire rebuilder on pipeline failures.