Skip to main content
Version: Tenzir v4.11

metrics

Retrieves metrics events from a Tenzir node.

Synopsis

metrics [--live]

Description

The metrics operator retrieves metrics events from a Tenzir node. Metrics events are collected every second.

--live

Work on all metrics events as they are generated in real-time instead of on metrics events persisted at a Tenzir node.

Schemas

Tenzir collects metrics with the following schemas.

tenzir.metrics.cpu

Contains a measurement of CPU utilization.

FieldTypeDescription
loadavg_1mdoubleThe load average over the last minute.
loadavg_5mdoubleThe load average over the last 5 minutes.
loadavg_15mdoubleThe load average over the last 15 minutes.

tenzir.metrics.disk

Contains a measurement of disk space usage.

FieldTypeDescription
pathstringThe byte measurements below refer to the filesystem on which this path is located.
total_bytesuint64The total size of the volume, in bytes.
used_bytesuint64The number of bytes occupied on the volume.
free_bytesuint64The number of bytes still free on the volume.

tenzir.metrics.memory

Contains a measurement of the available memory on the host.

FieldTypeDescription
total_bytesuint64The total available memory, in bytes.
used_bytesuint64The amount of memory used, in bytes.
free_bytesuint64The amount of free memory, in bytes.

tenzir.metrics.operator

Contains input and output measurements over some amount of time for a single operator instantiation.

FieldTypeDescription
pipeline_idstringThe ID of the pipeline where the associated operator is from.
runuint64The number of the run, starting at 1 for the first run.
hiddenboolTrue if the pipeline is running for the explorer.
operator_iduint64The ID of the operator inside the pipeline referenced above.
sourceboolTrue if this is the first operator in the pipeline.
transformationboolTrue if this is neither the first nor the last operator.
sinkboolTrue if this is the last operator in the pipeline.
internalboolTrue if the data flow is considered to internal to Tenzir.
timestamptimeThe time when this event was emitted (immediately after the collection period).
durationdurationThe timespan over which this data was collected.
starting_durationdurationThe time spent to start the operator.
processing_durationdurationThe time spent processing the data.
scheduled_durationdurationThe time that the operator was scheduled.
running_durationdurationThe time that the operator was running.
paused_durationdurationThe time that the operator was paused.
inputrecordMeasurement of the incoming data stream.
outputrecordMeasurement of the outgoing data stream.

The records input and output have the following schema:

FieldTypeDescription
unitstringThe type of the elements, which is void, bytes or events.
elementsuint64Number of elements that were seen during the collection period.
approx_bytesuint64An approximation for the number of bytes transmitted.

tenzir.metrics.process

Contains a measurement of the amount of memory used by the tenzir-node process.

FieldTypeDescription
current_memory_usageuint64The memory currently used by this process.
peak_memory_usageuint64The peak amount of memory, in bytes.
swap_space_usageuint64The amount of swap space, in bytes. Only available on Linux systems.

Examples

Show the CPU usage over the last hour:

metrics
| where #schema == "tenzir.metrics.cpu"
| where timestamp > 1 hour ago
| put timestamp, percent=loadavg_1m
Output
{
"timestamp": "2023-12-21T12:00:32.631102",
"percent": 0.40478515625
}
{
"timestamp": "2023-12-21T11:59:32.626043",
"percent": 0.357421875
}
{
"timestamp": "2023-12-21T11:58:32.620327",
"percent": 0.42578125
}
{
"timestamp": "2023-12-21T11:57:32.614810",
"percent": 0.50390625
}
{
"timestamp": "2023-12-21T11:56:32.609896",
"percent": 0.32080078125
}
{
"timestamp": "2023-12-21T11:55:32.605871",
"percent": 0.5458984375
}

Get the current memory usage:

metrics
| where #schema == "tenzir.metrics.memory"
| sort timestamp desc
| tail 1
| put current_memory_usage
Output
{
"current_memory_usage": 1083031552
}

Show the total pipeline ingress in bytes for every day over the last week, excluding pipelines run in the Explorer:

metrics
| where #schema == "tenzir.metrics.operator"
| where timestamp > 1 week ago
| where hidden == false and source == true
| summarize bytes=sum(output.approx_bytes) by timestamp resolution 1 day
Output
{
"timestamp": "2023-11-08T00:00:00.000000",
"bytes": 79927223
}
{
"timestamp": "2023-11-09T00:00:00.000000",
"bytes": 51788928
}
{
"timestamp": "2023-11-10T00:00:00.000000",
"bytes": 80740352
}
{
"timestamp": "2023-11-11T00:00:00.000000",
"bytes": 75497472
}
{
"timestamp": "2023-11-12T00:00:00.000000",
"bytes": 55497472
}
{
"timestamp": "2023-11-13T00:00:00.000000",
"bytes": 76546048
}
{
"timestamp": "2023-11-14T00:00:00.000000",
"bytes": 68643200
}

Show the three operator instantiations that produced the most events in total and their pipeline IDs:

metrics
| where #schema == "tenzir.metrics.operator"
| where output.unit == "events"
| summarize events=max(output.elements) by pipeline_id, operator_id
| sort events desc
| head 3
Output
{
"pipeline_id": "70a25089-b16c-448d-9492-af5566789b99",
"operator_id": 0,
"events": 391008694
}
{
"pipeline_id": "7842733c-06d6-4713-9b80-e20944927207",
"operator_id": 0,
"events": 246914949
}
{
"pipeline_id": "6df003be-0841-45ad-8be0-56ff4b7c19ef",
"operator_id": 1,
"events": 83013294
}

Get the disk usage over time:

metrics
| where #schema == "tenzir.metrics.disk"
| sort timestamp
| put timestamp, used_bytes
Output
{
"timestamp": "2023-12-21T12:52:32.900086",
"used_bytes": 461834444800
}
{
"timestamp": "2023-12-21T12:53:32.905548",
"used_bytes": 461834584064
}
{
"timestamp": "2023-12-21T12:54:32.910918",
"used_bytes": 461840302080
}
{
"timestamp": "2023-12-21T12:55:32.916200",
"used_bytes": 461842751488
}

Get the memory usage over time:

metrics
| where #schema == "tenzir.metrics.memory"
| sort timestamp
| put timestamp, used_bytes
Output
{
"timestamp": "2023-12-21T13:08:32.982083",
"used_bytes": 48572645376
}
{
"timestamp": "2023-12-21T13:09:32.986962",
"used_bytes": 48380682240
}
{
"timestamp": "2023-12-21T13:10:32.992494",
"used_bytes": 48438878208
}
{
"timestamp": "2023-12-21T13:11:32.997889",
"used_bytes": 48491839488
}
{
"timestamp": "2023-12-21T13:12:33.003323",
"used_bytes": 48529952768
}