sample
Dynamically samples events from a event stream.
Synopsis
sample [--period <period>] [--mode <mode>] [--min-events <uint>] [--max-rate <uint>] [--max-samples <uint>]
Description
Dynamically samples input data from a stream based on the frequency of receiving events for streams with varying load.
The operator counts the number of events received in the period
and applies
the specified function on the count to calculate the sampling rate for the next
period.
--period <period>
The duration to count events in, i.e., how often the sample rate is computed.
The sampling rate for the first window is 1:1
.
Defaults to 30 seconds
.
--mode <mode>
The function used to compute the sampling rate:
"ln"
(default)"log2"
"log10"
"sqrt"
--min-events <uint>
The minimum number of events that must be received during the previous sampling
period for the sampling mode to be applied in the current period. If the number
of events in a sample group falls below this threshold, a 1:1
sample rate is
used instead.
Defaults to 30
.
--max-rate <uint>
The sampling rate is capped to this value if the computed rate is higher than this.
--max-samples <uint>
The maximum number of events to emit per period
.
Examples
Sample a feed log-stream
every 30s dynamically, only changing rate when more
than 50 events (min-events
) are received. Additionally, cap the max sampling
rate to 1:500
, i.e. 1 sample for every 500 events or more (max-rate
).
subscribe "log-stream"
| sample --period 30s --min-events=50 --max-rate=500
Sample some metrics
every hour, limiting the max samples per period to 5000
samples (max-samples
) and limiting the overall sample count to 100k samples
(head
).
subscribe "metrics"
| sample --period 1h --max-samples 5k | head 100k