Download the release on GitHub.
Features
Section titled “Features”Always convert JSON null to VAST nil
Section titled “Always convert JSON null to VAST nil”The default schema for Suricata has been updated to support the suricata.ftp
and suricata.ftp_data event types.
VAST now prints the location of the configuration file that is used.
By @dominiklohmann in #1009.
Protect DB directory with PID lock
Section titled “Protect DB directory with PID lock”VAST now writes a PID lock file on startup to prevent multiple server processes
from accessing the same persistent state. The pid.lock file resides in the
vast.db directory.
Changes
Section titled “Changes”Rename vector to list
Section titled “Rename vector to list”The vector type has been renamed to list. In an effort to streamline the
type system vocabulary, we favor list over vector because it’s closer to
existing terminology (e.g., Apache Arrow). This change requires updating
existing schemas by changing vector<T> to list<T>.
Allow the ’-’ in the expression key parser
Section titled “Allow the ’-’ in the expression key parser”The expression field parser now allows the ’-’ character.
Remove set data type
Section titled “Remove set data type”The set type has been removed. Experience with the data model showed that
there is no strong use case to separate sets from vectors in the core. While
this may be useful in programming languages, VAST deals with immutable data
where set constraints have been enforced upstream. This change requires updating
existing schemas by changing set<T> to vector<T>. In the query language, the
new symbol for the empty map changed from {-} to {}, as it now
unambiguously identifies map instances.
Expand CAF stream slot ids to 32 bits
Section titled “Expand CAF stream slot ids to 32 bits”We now bundle a patched version of CAF, with a changed ABI. This means that if you’re linking against the bundled CAF library, you also need to distribute that library so that VAST can use it at runtime. The versions are API compatible so linking against a system version of CAF is still possible and supported.
Bug Fixes
Section titled “Bug Fixes”Fix bug in decoding multi-object MsgPack types
Section titled “Fix bug in decoding multi-object MsgPack types”MessagePack-encoded table slices now work correctly for nested container types.
Expand CAF stream slot ids to 32 bits
Section titled “Expand CAF stream slot ids to 32 bits”When running VAST under heavy load, CAF stream slot ids could wrap around after a few days and deadlock the system. As a workaround, we extended the slot id bit width to make the time until this happens unrealistically large.
Allow the ’-’ in the expression key parser
Section titled “Allow the ’-’ in the expression key parser”A bug in the expression parser prevented the correct parsing of fields starting with either ‘F’ or ‘T’.
Terminate exporters when sinks die
Section titled “Terminate exporters when sinks die”When continuous query in a client process terminated, the node did not clean up the corresponding server-side state. This memory leak no longer exists.
Support hard-kill for unresponsive actors
Section titled “Support hard-kill for unresponsive actors”The shutdown process of the server process could potentially hang forever. VAST
now uses a 2-step procedure that first attempts to terminate all components
cleanly. If that fails, it will attempt a hard kill afterwards, and if that
fails after another timeout, the process will call abort(3).
Make port-encoding for Arrow host-independent
Section titled “Make port-encoding for Arrow host-independent”The port encoding for Arrow-encoded table slices is now host-independent and always uses network-byte order.
By @dominiklohmann in #1007.
Close file descriptor by default in ‘vast::file’
Section titled “Close file descriptor by default in ‘vast::file’”Some file descriptors remained open when they weren’t needed any more. This descriptor leak has been fixed.
Always convert JSON null to VAST nil
Section titled “Always convert JSON null to VAST nil”Importing JSON no longer fails for JSON fields containing null when the
corresponding VAST type in the schema is a non-trivial type like
vector<string>.
By @dominiklohmann in #1009.
Correct the use of ::read()
Section titled “Correct the use of ::read()”Incomplete reads have not been handled properly, which manifested for files larger than 2GB. On macOS, writing files larger than 2GB may have failed previously. VAST now respects OS-specific constraints on the maximum block size.
Shutdown node when component startup fails
Section titled “Shutdown node when component startup fails”VAST did not terminate when a critical component failed during startup. VAST now binds the lifetime of the node to all critical components.
Don’t overwrite index state after startup error
Section titled “Don’t overwrite index state after startup error”VAST would overwrite existing on-disk state data when encountering a partial read during startup. This state-corrupting behavior no longer exists.