Skip to main content
Version: Tenzir v4.5

Metrics

Tenzir collects various metrics during execution. This reference describes what metrics are available and what they mean. We describe how to collect metrics in the corresponding user guide.

For the reference table below, these symbols indicate presense of additional metadata:

SymbolKeyValue
🔎queryA UUID to identify the query.
🪪issuerA human-readable identifier of the query issuer.
💽partition-typeOne of "active" or "passive".
#️⃣partition-versionThe internal partition version.
💾store-typeOne of "parquet" or "feather".
🗂️schemaThe schema name.

For all keys that show throughput rates in #events/second, e.g., <component>.rate, the keys <component>.events and <component>.duration are dividend and divisor, respectively. They are not listed explicitly in the below table.

Generally, counts are reset after a component sends out a telemetry report. E.g., the total number of invalid lines the JSON parser encountered is reflected by the sum of all json-reader.invalid-line events.

Reference

KeyDescriptionUnitMetadata
accountant.startupThe first event in the lifetime of Tenzir.constant 0
accountant.shutdownThe last event in the lifetime of Tenzir.constant 0
archive.rateThe rate of events processed by the archive component.#events/second
arrow-writer.rateThe rate of events processed by the Arrow sink.#events/second
ascii-writer.rateThe rate of events processed by the ascii sink.#events/second
csv-reader.rateThe rate of events processed by the CSV source.#events/second
csv-writer.rateThe rate of events processed by the CSV sink.#events/second
importer.rateThe rate of events processed by the importer component.#events/second
index.memory-usageThe rough estimate of memory used by the index#bytes
ingest.rateThe ingest rate keyed by the schema name.#events/second🗂️
ingest-total.rateThe total ingest rate of all schemas.#events/second
json-reader.invalid-lineThe number of invalid NDJSON lines.#events
json-reader.rateThe rate of events processed by the JSON source.#events/second
json-reader.unknown-layoutThe number if NDJSON lines with an unknown layout.#event
json-writer.rateThe rate of events processed by the JSON sink.#events/second
catalog.lookup.candidatesThe number of candidate partitions considered for a query.#partitions🔎🪪
catalog.lookup.runtimeThe duration of a query evaluation in the catalog.#milliseconds🔎🪪
catalog.lookup.hitsThe number of results of a query in the catalog.#events🔎🪪
catalog.memory-usageThe rough estimate of memory used by the catalog#bytes
catalog.num-partitionsThe number of partitions registered in the catalog per schema.#partitions🗂️#️⃣
catalog.num-eventsThe number of events registered in the catalog per schema.#events🗂️#️⃣
catalog.num-partitions-totalThe sum of all partitions registered in the catalog.#partitions
catalog.num-events-totalThe sum of all events registered in the catalog.#events
node_throughput.rateThe rate of events processed by the node component.#events/second
null-writer.rateThe rate of events processed by the null sink.#events/second
partition.events-writtenThe number of events written in one partition.#events🗂
partition.lookup.runtimeThe duration of a query evaluation in one partition.#milliseconds🔎🪪💽
partition.lookup.hitsThe number of results of a query in one partition.#events🔎🪪💽
pcap-reader.discard-rateThe rate of packets discarded.#events-dropped/#events-received
pcap-reader.discardThe number of packets discarded by the reader.#events
pcap-reader.drop-rateThe rate of packets dropped.#events-dropped/#events-received
pcap-reader.dropThe number of packets dropped by the reader.#events
pcap-reader.ifdropThe number of packets dropped by the network interface.#events
pcap-reader.rateThe rate of events processed by the PCAP source.#events/second
pcap-reader.recvThe number of packets received.#events
pcap-writer.rateThe rate of events processed by the PCAP sink.#events/second
rebuilder.partitions.remainingThe number of partitions scheduled for rebuilding.#partitions
rebuilder.partitions.rebuildingThe number of partitions currently being rebuilt.#partitions
rebuilder.partitions.completedThe number of partitions rebuilt in the current run.#partitions
scheduler.backlog.customThe number of custom priority queries in the backlog.#queries
scheduler.backlog.lowThe number of low priority queries in the backlog.#queries
scheduler.backlog.normalThe number of normal priority queries in the backlog.#queries
scheduler.backlog.highThe number of high priority queries in the backlog.#queries
scheduler.partition.current-lookupsThe number of partition lookups that are currently running.#workers
scheduler.partition.lookupsQuery lookups executed on individual partitions.#partition-lookups
scheduler.partition.materializationsPartitions loaded from disk.#partitions
scheduler.partition.pendingThe number of queued partitions.#partitions
scheduler.partition.remaining-capacityThe number of partition lookups that could be scheduled immediately.#workers
scheduler.partition.scheduledThe number of scheduled partitions.#partitions
active-store.lookup.runtimeThe number of results of a query in an active store.#events🔎🪪💾
active-store.lookup.hitsThe number of results of a query in an active store.#events🔎🪪💾
passive-store.lookup.runtimeThe number of results of a query in a passive store.#events🔎🪪💾
passive-store.lookup.hitsThe number of results of a query in a passive store.#events🔎🪪💾
passive-store.init.runtimeTime until the store is ready serve queries.nanoseconds💾
posix-filesystem.checks.failedThe number of failed file checks since process start.
posix-filesystem.checks.successfulThe number of successful file checks since process start.
posix-filesystem.erases.bytesThe number of bytes erased since process start.#bytes
posix-filesystem.erases.failedThe number of failed file erasures since process start.
posix-filesystem.erases.successfulThe number of successful file erasures since process start.
posix-filesystem.mmaps.bytesThe number of bytes memory-mapped since process start.#bytes
posix-filesystem.mmaps.failedThe number of failed file memory-maps since process start.
posix-filesystem.mmaps.successfulThe number of successful file memory-maps since process start.
posix-filesystem.moves.failedThe number of failed file moves since process start.
posix-filesystem.moves.successfulThe number of successful file moves since process start.
posix-filesystem.reads.bytesThe number of bytes read since process start.#bytes
posix-filesystem.reads.failedThe number of success file reads since process start.
posix-filesystem.reads.successfulThe number of success file reads since process start.
posix-filesystem.writes.bytesThe number of bytes written since process start.#bytes
posix-filesystem.writes.failedThe number of failed file writes since process start.
posix-filesystem.writes.successfulThe number of successful file writes since process start.
source.startTimepoint when the source started.nanoseconds since epoch
source.stopTimepoint when the source stopped.nanoseconds since epoch
syslog-reader.rateThe rate of events processed by the syslog source.#events/second
test-reader.rateThe rate of events processed by the test source.#events/second
zeek-reader.rateThe rate of events processed by the Zeek source.#events/second