Send to destinations

This guide shows you how to send data to various destinations using TQL output operators. You’ll learn about destination operators, file output patterns, and expression-based serialization.

Destination operators

TQL provides to_* operators for sending events to various destinations. These operators accept expressions for flexible serialization.

Message brokers

Send events to message brokers like Kafka, Amazon Kinesis Data Streams, and NATS.

Send to Kafka with automatic JSON formatting:

subscribe "security-events"
to_kafka "events"

Specify explicit serialization with the message parameter:

subscribe "logs"
to_kafka "events", message=this.print_json()

The message parameter accepts any expression that evaluates to a string or blob.

Send to NATS JetStream:

subscribe "security-events"
to_nats "alerts"

The NATS server must have a JetStream stream that captures the subject you publish to.

Send to Kinesis with the default NDJSON serialization:

subscribe "security-events"
to_amazon_kinesis "security-events"

Analytics platforms

Send data to platforms like Splunk, OpenSearch, and Elasticsearch.

Send to a Splunk HEC endpoint:

subscribe "logs"
to_splunk "https://splunk.example.com:8088",
  hec_token=secret("SPLUNK_HEC_TOKEN")

Send to OpenSearch with index routing:

subscribe "security"
to_opensearch "https://opensearch.example.com:9200",
  action="index",
  index="security-events"

Cloud services

Route events to cloud destinations like SQS and Cloud Pub/Sub.

Send to SQS:

subscribe "notifications"
to_amazon_sqs "sqs://notifications", message=this.print_json()

Send to Pub/Sub:

subscribe "events"
to_google_cloud_pubsub project_id="my-project", topic_id="events", message=this.print_ndjson()

File output

For writing to files, use a to_* destination operator with a printing subpipeline. This pattern separates serialization from storage.

Write JSON to a local file:

subscribe "logs"
to_file "output.json" {
  write_json
}

Write compressed Parquet:

export
to_file "archive.parquet.zst" {
  write_parquet
}

Write JSON Lines to S3:

to_file "s3://bucket/logs/events.jsonl" {
  write_json
}

Send newline-delimited JSON over TCP:

to_tcp "collector.example.com:5044" { write_ndjson }

For protocols that expect a delimiter after every message, print the event to a string and use write_delimited for the byte-stream framing:

to_tcp "collector.example.com:12201" {
  write_delimited this.print_ndjson(strip_null_fields=true), "\x00"
}

Expression-based serialization

Destination operators use expressions for flexible message formatting:

Serialize the entire event

Use the default event serialization:

to_kafka "events"

Serialize as compact JSON without nulls:

to_kafka "events", message=this.print_json(include_nulls=false)

Serialize specific fields

Send only a specific field:

to_kafka "alerts", message=alert_message

Combine fields into a formatted string:

to_kafka "metrics", message=f"{host}: {metric_name}={value}"

Dynamic routing

Route events to different destinations based on content:

to_kafka f"events.{event_type}"