Unreleased changes.
Features
Section titled “Features”from_azure_blob_storage operator
Section titled “from_azure_blob_storage operator”The new from_azure_blob_storage operator works similarly to from_file but
supports additional Azure Blob Storage specific options.
For example, you can set the account_key:
from_azure_blob_storage "abfs://container/data/*.csv", account_key="your-account-key"Dynamic namespace and retry logic for to_google_secops
Section titled “Dynamic namespace and retry logic for to_google_secops”The to_google_secops operator now retries requests which fail with a 5XX or
a 429 status code. Additionally, the namespace option of the operator now
supports all expressions that evaluate to a string.
Bug Fixes
Section titled “Bug Fixes”Misleading from_file remove=true warning
Section titled “Misleading from_file remove=true warning”The from_file operator emits a warning when using remove=true if the file
could not be removed. When deleting the last file inside an S3 directory, we
keep that directory around by inserting a zero-sized object. However, this
failed when the necessary PutObject permissions were not granted, thus
emitting a warning even though the file was removed successfully. For this
specific case, we thus no longer emit a warning. Other issues during file
deletion are still reported.
Timezone troubles from parse_time()
Section titled “Timezone troubles from parse_time()”We fixed assertion failures when using the parse_time function with the %z
or %Z specifiers.
Fixed crash when writing out enumerations
Section titled “Fixed crash when writing out enumerations”We fixed a rare crash that could occur when writing/printing enumeration values in various formats.
By @IyeOnline in #5434.
Buffering in the fork operator
Section titled “Buffering in the fork operator”We fixed an issue in the fork operator where the last event would get stuck.