Collect metrics

Tenzir keeps track of metrics about node resource usage, pipeline state, and runtime performance.

Metrics are stored as internal events in the node’s storage engine, allowing you to work with metrics just like regular data. Use the metrics input operator to access the metrics. The operator documentation lists all available metrics in detail.

The metrics operator provides a copy of existing metrics. You can use it multiple time to reference the same metrics feed.

Write metrics to a file

Export metrics continuously to a file via metrics --live:

metrics live=true
to_file "metrics.json", append=true {
  write_ndjson
}

This attaches to incoming metrics feed, renders them as NDJSON, and then writes the output to a file. Without the live option, the metrics operator returns the snapshot of all historical metrics.

Summarize metrics

You can transform metrics like ordinary data, e.g., write aggregations over metrics to compute runtime statistics suitable for reporting or dashboarding:

metrics "operator"
where sink == true
summarize runtime=sum(duration), pipeline_id
sort -runtime

The above example computes the total runtime over all pipelines grouped by their unique ID.

Inspect pipeline throughput

Use tenzir.metrics.pipeline events to compare the number of events and bytes that enter and leave each pipeline:

metrics "pipeline"
summarize ingress_events=sum(ingress.events), ingress_bytes=sum(ingress.bytes), egress_events=sum(egress.events), egress_bytes=sum(egress.bytes), pipeline_id
sort -egress_events

The events fields count records that passed through the pipeline during the metric period. The bytes fields approximate the amount of data transferred.