Download the release on GitHub.
Features
Section titled “Features”Allow output format selection for the pivot/explore command
Section titled “Allow output format selection for the pivot/explore command”The output format for the explore and pivot commands can now be set using
the explore.format and pivot.format options respectively. Both default to
JSON.
By @dominiklohmann in #921.
Support type relaxation for JSON import
Section titled “Support type relaxation for JSON import”The import json command’s type restrictions are more relaxed now, and can
additionally convert from JSON strings to VAST internal data types.
By @dominiklohmann in #891.
Support /etc/vast/vast.conf as global config
Section titled “Support /etc/vast/vast.conf as global config”VAST now supports /etc/vast/vast.conf as an additional fallback for the
configuration file. The following file locations are looked at in order: Path
specified on the command line via --config=path/to/vast.conf, vast.conf in
current working directory, ${INSTALL_PREFIX}/etc/vast/vast.conf, and
/etc/vast/vast.conf.
By @dominiklohmann in #898.
Support aging out data based on a query
Section titled “Support aging out data based on a query”VAST now supports aging out existing data. This feature currently only concerns
data in the archive. The options system.aging-frequency and
system.aging-query configure a query that runs on a regular schedule to
determine which events to delete. It is also possible to trigger an aging cycle
manually.
By @dominiklohmann in #929.
Add limit options for vast explore
Section titled “Add limit options for vast explore”VAST now has options to limit the amount of results produced by an invocation of
vast explore.
Forcefully emit batches on input timeout error
Section titled “Forcefully emit batches on input timeout error”The import command gained a new --read-timeout option that forces data to be
forwarded to the importer regardless of the internal batching parameters and
table slices being unfinished. This allows for reducing the latency between the
import command and the node. The default timeout is 10 seconds.
By @dominiklohmann in #916.
Add Bloom filter meta index
Section titled “Add Bloom filter meta index”The meta index now uses Bloom filters for equality queries involving IP addresses. This especially accelerates queries where the user wants to know whether a certain IP address exists in the entire database.
Changes
Section titled “Changes”Rename the ‘default’ table slice type to ‘caf’
Section titled “Rename the ‘default’ table slice type to ‘caf’”The default table slice type has been renamed to caf. It has not been the
default when built with Apache Arrow support for a while now, and the new name
more accurately reflects what it is doing.
By @dominiklohmann in #948.
Print timestamps with full precision for JSON
Section titled “Print timestamps with full precision for JSON”The JSON export format now renders timestamps using strings instead of numbers in order to avoid possible loss of precision.
By @dominiklohmann in #909.
PRs 908-951
Section titled “PRs 908-951”The options system.table-slice-type and system.table-slice-size have been
removed, as they duplicated import.table-slice-type and
import.table-slice-size respectively.
Bug Fixes
Section titled “Bug Fixes”Improve handling of UTF-8 input
Section titled “Improve handling of UTF-8 input”The export json command now correctly unescapes its output.
VAST now correctly checks for control characters in inputs.
By @dominiklohmann in #910.
Simplify id space management
Section titled “Simplify id space management”A bogus import process that assembled table slices with a greater number of events than expected by the node was able to lead to wrong query results.
Fix use-after-free bug in indexer state
Section titled “Fix use-after-free bug in indexer state”A use after free bug would sometimes crash the node while it was shutting down.