Skip to main content
Version: v4.21

Glossary

This page defines central terms in the Tenzir ecosystem.

missing term?

If you are missing a term, please open a GitHub Discussion or ping us in our Discord chat.

App​

Web user interface to access platform at app.tenzir.com.

The app is a web application that partially runs in the user's browser. It is written in Svelte.

Catalog​

Maintains partition ownership and metadata.

The catalog is a component in the node that owns the partitions, keeps metadata about them, and maintains a set of sparse secondary indexes to identify relevant partitions for a given query. It offers a transactional interface for adding and removing partitions.

Connector​

Manages chunks of raw bytes by interacting with a resource.

A connector is either a loader that acquires bytes from a resource, or a saver that sends bytes to a resource. Example connectors are file, kafka, and nic.

Context​

A stateful object used for in-band enrichment.

Contexts live inside a node and you can manage them with the context operator. A context has pluggable type, such as a lookup table, GeoIP database, or a custom plugin. The enrich places a context into a pipeline for enrichment.

Format​

Translates between bytes and events.

A format is either a parser that converts bytes to events, or a printer that converts events to bytes. Example formats are json, cef, and pcap.

Index​

Optional data structures for accelerating historical queries.

Tenzir has sparse indexes. Sparse indexes live in memory and point to partitions.

Integration​

A set of pipelines to integrate with a third-party product.

An integration describes use cases in combination with a specific product or tool. Based on the depth of the configuration, this may require configuration on either end.

Loader​

A connector that acquires bytes.

A loader is the dual to a saver. It has a no input and only performs a side effect that acquires bytes. Use a loader in the from or load operators.

Node​

A host for pipelines and storage reachable over the network.

The tenzir-node binary starts a node in a dedicated server process that listens on TCP port 5158.

Metrics​

Runtime statistics about pipeline execution.

Operator​

The building block of a pipeline.

An operator is a source, transformation, or sink.

PaC​

The acronym PaC stands for Pipelines as Code. It is meant as an adaptation of Infrastructure as Code (IaC) with pipelines represent the (data) infrastructure that is provisioning as code.

Parser​

A format that translates bytes into events.

A parser is the dual to a printer. Use a parser in the from or read operators. You can use the parse operator to parse a single field with a parser.

Partition​

The horizontal scaling unit of the storage attached to a node.

A partition contains the raw data and optionally a set of indexes. Supported formats are Parquet or Feather.

Pipeline​

Combines a set of operators into a dataflow graph.

Platform​

Control plane for nodes and pipelines, accessible through app at app.tenzir.com.

Printer​

A format that translates events into bytes.

A printer is the dual to a parser. Use a parser in the to or write operators.

Saver​

A connector that emits bytes.

A saver is the dual to a loader. It has a no output and only performs a side effect that emits bytes. Use a saver in the to or save operators.

Schema​

A named record type describing the top-level structure of a data frame.

Schemas

Sink​

An operator consuming input, without producing any output.

Source​

An operator producing output, without consuming any input.

TQL​

An acronym for Tenzir Query Language.

TQL is the language in which users write pipelines.

Transformation​

An operator consuming both input and producing output.