This release adds a new Azure Blob Storage operator with account key authentication and improves Google Security Operations retry handling. It also contains various small fixes and improvements.
Download the release on GitHub.
Features
Section titled “Features”from_azure_blob_storage operator
Section titled “from_azure_blob_storage operator”The new from_azure_blob_storage operator works similarly to from_file but
supports additional Azure Blob Storage specific options.
For example, you can set the account_key:
from_azure_blob_storage "abfs://container/data/*.csv", account_key="your-account-key"Dynamic namespace and retry logic for to_google_secops
Section titled “Dynamic namespace and retry logic for to_google_secops”The to_google_secops operator now retries requests which fail with a 5XX or
a 429 status code. Additionally, the namespace option of the operator now
supports all expressions that evaluate to a string.
Bug Fixes
Section titled “Bug Fixes”Misleading from_file remove=true warning
Section titled “Misleading from_file remove=true warning”The from_file operator emits a warning when using remove=true if the file
could not be removed. When deleting the last file inside an S3 directory, we
keep that directory around by inserting a zero-sized object. However, this
failed when the necessary PutObject permissions were not granted, thus
emitting a warning even though the file was removed successfully. For this
specific case, we thus no longer emit a warning. Other issues during file
deletion are still reported.
Timezone troubles from parse_time()
Section titled “Timezone troubles from parse_time()”We fixed assertion failures when using the parse_time function with the %z
or %Z specifiers.
Fixed crash when writing out enumerations
Section titled “Fixed crash when writing out enumerations”We fixed a rare crash that could occur when writing/printing enumeration values in various formats.
By @IyeOnline in #5434.
Buffering in the fork operator
Section titled “Buffering in the fork operator”We fixed an issue in the fork operator where the last event would get stuck.