Skip to main content
Version: Next

Glossary

This page defines central terms in the Tenzir ecosystem.

missing term?

If you are missing a term, please open a GitHub Discussion or ping us in our Discord chat.

App​

Web user interface to access platform at app.tenzir.com.

The app is a web application that partially runs in the user's browser. It is written in Svelte.

Catalog​

Maintains partition ownership and metadata.

The catalog is a component in the node that owns the partitions, keeps metadata about them, and maintains a set of sparse secondary indexes to identify relevant partitions for a given query. It offers a transactional interface for adding and removing partitions.

Connector​

Manages chunks of raw bytes by interacting with a resource.

A connector is either a loader that acquires bytes from a resource, or a saver that sends bytes to a resource. Loaders are implemented as ordinary operators prefixed with load_* while savers are prefixed with save_*.

Context​

A stateful object used for in-band enrichment.

Contexts come in various types, such as a lookup table, Bloom filter, and GeoIP database. They live inside a node and you can enrich with them in other pipelines.

Destination​

An pipeline ending with an output operator preceded by a subscribe input operator.

Format​

Translates between bytes and events.

A format is either a parser that converts bytes to events, or a printer that converts events to bytes. Example formats are json, cef, and pcap.

Index​

Optional data structures for accelerating historical queries.

Tenzir has sparse indexes. Sparse indexes live in memory and point to partitions.

Input​

An operator that only producing data, without consuming anything.

Integration​

A set of pipelines to integrate with a third-party product.

An integration describes use cases in combination with a specific product or tool. Based on the depth of the configuration, this may require configuration on either end.

Library​

A collection of packages.

Our community library is freely available at GitHub.

Loader​

A connector that acquires bytes.

A loader is the dual to a saver. It has a no input and only performs a side effect that acquires bytes. Use a loader implicitly with the from operator or explicitly with the load_* operators.

Node​

A host for pipelines and storage reachable over the network.

The tenzir-node binary starts a node in a dedicated server process that listens on TCP port 5158.

Metrics​

Runtime statistics about pipeline execution.

OCSF​

The Open Cybersecurity Schema Framework (OCSF) is a cross-vendor schema for security event data. Our community library contains packages that map data sources to OCSF.

Operator​

The building block of a pipeline.

An operator is a input, transformation, or output.

Output​

An operator consuming data, without producing anything.

PaC​

The acronym PaC stands for Pipelines as Code. It is meant as an adaptation of Infrastructure as Code (IaC) with pipelines represent the (data) infrastructure that is provisioning as code.

Package​

A collection of pipelines and contexts.

Parser​

A bytes-to-events operator.

A parser is the dual to a printer. Use a parser implicitly in the from operator.

Partition​

The horizontal scaling unit of the storage attached to a node.

A partition contains the raw data and optionally a set of indexes. Supported formats are Parquet or Feather.

Pipeline​

Combines a set of operators into a dataflow graph.

Platform​

Control plane for nodes and pipelines, accessible through app at app.tenzir.com.

Printer​

An events-to-bytes operator.

A format that translates events into bytes.

A printer is the dual to a parser. Use a parser implicitly in the to operator.

Saver​

A connector that emits bytes.

A saver is the dual to a loader. It has a no output and only performs a side effect that emits bytes. Use a saver implicitly with the to operator or explicitly with the save_* operators.

Schema​

A top-level record type of an event.

Source​

An pipeline starting with an input operator followed by a publish output operator.

TQL​

An acronym for Tenzir Query Language.

TQL is the language in which users write pipelines.

Transformation​

An operator consuming both input and producing output.