Download the release on GitHub.
Features
Section titled “Features”Cloud matchers
Section titled “Cloud matchers”We can now use matchers in AWS using the vast-cloud CLI matcher plugin.
Make the connection timeout configurable
Section titled “Make the connection timeout configurable”The new vast.connection-timeout option allows for configuring the timeout VAST
clients use when connecting to a VAST server. The value defaults to 10s; setting
it to a zero duration causes produces an infinite timeout.
By @dominiklohmann in #2499.
Make the rebuild command more reliable
Section titled “Make the rebuild command more reliable”VAST now continuously rebuilds outdated and merges undersized partitions in the
background. The new option vast.automatic-rebuild controls how many resources
to spend on this. To disable this behavior, set the option to 0; the default is
1.
Rebuilding now emits metrics under the keys
rebuilder.partitions.{remaining,rebuilding,completed}. The vast status rebuild command additionally shows information about the ongoing rebuild.
By @dominiklohmann in #2493.
Implement optional dense indexes
Section titled “Implement optional dense indexes”VAST’s partition indexes are now optional, allowing operators to control the trade-off between disk-usage and query performance for every field.
Changes
Section titled “Changes”Lower the impact of low-priority queries
Section titled “Lower the impact of low-priority queries”We improved the operability of VAST servers under high load from automated
low-priority queries. VAST now considers queries issued with --low-priority,
such as automated retro-match queries, with even less priority compared to
regular queries (down from 33.3% to 4%) and internal high-priority queries used
for rebuilding and compaction (down from 12.5% to 1%).
By @dominiklohmann in #2484.
Make the rebuild command more reliable
Section titled “Make the rebuild command more reliable”The default value for vast.active-partition-timeout is now 5 minutes (down
from 1 hour), causing VAST to persist underful partitions earlier.
We split the vast rebuild command into two: vast rebuild start and vast rebuild stop. Rebuild orchestration now runs server-side, and only a single
rebuild may run at a given time. We also made it more intuitive to use:
--undersized now implies --all, and a new --detached option allows for
running rebuilds in the background.
By @dominiklohmann in #2493.
Bug Fixes
Section titled “Bug Fixes”Respect --connection-timeout in more places
Section titled “Respect --connection-timeout in more places”Configuration options representing durations with an associated command-line
option like vast.connection-timeout and --connection-timeout were not picked
up from configuration files or environment variables. This now works as
expected.
By @dominiklohmann in #2503.
Add active partition actor to unpersisted partitions on decomission
Section titled “Add active partition actor to unpersisted partitions on decomission”VAST no longer occasionally prints warnings about no longer available partitions when queries run concurrently to imports.
Implement a flatbuffer container class to hold excess table slices in segments
Section titled “Implement a flatbuffer container class to hold excess table slices in segments”VAST can now store data in segments bigger than 2GiB in size each.
VAST can now store column indexes that are bigger than 2GiB.
Implement optional dense indexes
Section titled “Implement optional dense indexes”VAST properly processes queries for fields with skip attribute.
Wait until all stores have exited before finishing a partition transform
Section titled “Wait until all stores have exited before finishing a partition transform”Fixed a race condition where the output of a partition transform
could be reused before it was fully written to disk, for example
when running vast rebuild.
Display store load failures to the user
Section titled “Display store load failures to the user”Partitions now fail early when their stores fail to load from disk, detailing what went wrong in an error message.
By @dominiklohmann in #2507.
Activate cloud plugins explicitely
Section titled “Activate cloud plugins explicitely”We changed the way vast-cloud is loading its cloud plugins to make it more
explicit. This avoids inconsitent defaults assigned to variables when using core
commands on specific plugins.
Don’t abort startup if individual partitions fail to load
Section titled “Don’t abort startup if individual partitions fail to load”The rebuild command, automatic rebuilds, and compaction are now much faster,
and match the performance of the import command for building indexes.