Glossary

This page defines central terms in the Tenzir ecosystem.

missing term?

If you are missing a term, please open a GitHub Discussion or ping us in our Discord chat.

App

Web user interface to access platform at app.tenzir.com.

The app is a web application that partially runs in the user's browser. It is written in Svelte.

• Use the app

Catalog

Maintains partition ownership and metadata.

The catalog is a component in the node that owns the partitions, keeps metadata about them, and maintains a set of sparse secondary indexes to identify relevant partitions for a given query. It offers a transactional interface for adding and removing partitions.

• Tune catalog fragmentation
• Configure the catalog

Connector

Manages chunks of raw bytes by interacting with a resource.

A connector is either a loader that acquires bytes from a resource, or a saver that sends bytes to a resource. Example connectors are file, kafka, and nic.

• See all available connectors

Format

Translates between bytes and events.

A format is either a parser that converts bytes to events, or a printer that converts events to bytes. Example formats are json, cef, and pcap.

• See all available formats

Index

Optional data structures for accelerating historical queries.

Tenzir has sparse indexes. Sparse indexes live in memory and point to partitions.

• Configure the catalog

Node

A host for pipelines and storage reachable over the network.

The tenzir-node binary starts a node in a dedicated server process that listens on TCP port 5158.

• Deploy a node
• Use the app to manage a node
• Use the REST API to manage a node
• Import into a node
• Export from a node

Metrics

Runtime statistics about pipeline execution.

• Collect metrics

Operator

The building block of a pipeline.

An operator is a source, transformation, or sink.

• See all available operators

Partition

The horizontal scaling unit of the storage attached to a node.

A partition contains the raw data and optionally a set of indexes. Supported formats are Parquet or Feather.

• Control the partition size
• Configure catalog and partition indexes
• Select the store format
• Adjust the store compression
• Rebuild partitions

Pipeline

Combines a set of operators into a dataflow graph.

• Understand how pipelines work
• Understand the pipeline language
• Run a pipeline

Platform

Control plane for nodes and pipelines, accessible through app at app.tenzir.com.

• Use the app

Schema

A named record type describing the top-level structure of a data frame.

Schemas

• Show available schemas

Sink

An operator consuming input, without producing any output.

• See all available sinks

Source

An operator producing output, without consuming any input.

• See all available sources

TQL

An acronym for Tenzir Query Language.

TQL is the language in which users write pipelines.

• Learn more about the language

Transformation

An operator consuming both input and producing output.

• See all transformations