ZeroMQ

ZeroMQ (0mq) is a light-weight messaging framework with various socket types. Tenzir supports writing to PUB sockets and reading from SUB sockets, both in bind mode and connect mode.

Use the IP address 0.0.0.0 to listen on all available network interfaces.

The event-oriented ZeroMQ operators are:

• from_zmq: Connects as a SUB socket and receives events.
• accept_zmq: Binds a SUB socket and receives events.
• to_zmq: Connects as a PUB socket and sends events.
• serve_zmq: Binds a PUB socket and sends events.

Tenzir documents these operators for PUB/SUB-style use. ZeroMQ itself does not have a first-class topic abstraction. Instead, Tenzir uses an optional prefix that is prepended to outgoing messages and matched by subscribers with ZeroMQ’s native byte-prefix filtering. Receivers strip the matched prefix before running their nested read_* pipeline unless keep_prefix=true.

Because ZeroMQ is entirely asynchronous, publishers send messages even when no subscriber is present. This can lead to lost messages when the publisher begins operating before the subscriber. To avoid data loss due to such races, pass monitor=true on to_zmq or serve_zmq to wait until at least one remote peer has connected on TCP transports.

Examples

Connect to a remote publisher and parse JSON

from_zmq "tcp://collector.example.com:5555" {
  read_json
}

Receive a prefixed stream

accept_zmq "tcp://0.0.0.0:5555", prefix="alerts/" {
  read_ndjson
}

Publish events with a dynamic prefix

export
serve_zmq "tcp://0.0.0.0:5555", encoding="json", prefix=kind + "/"

Connect and publish JSON

export
to_zmq "tcp://collector.example.com:5555", encoding="json"