load_tcp

Loads bytes from a TCP or TLS connection.

load_tcp endpoint:string, [parallel=int, peer_field=field, tls=record,
                           max_buffered_chunks=int { … }]

Description

Reads bytes from the given endpoint via TCP or TLS.

endpoint: string

The endpoint at which the server will listen. Must be of the form [tcp://]<hostname>:<port>. Use the hostname 0.0.0.0 to accept connections on all interfaces.

parallel = int (optional)

Number of threads to use for reading from connections.

Defaults to 1.

peer_field = field (optional)

Write a record with the fields ip, port, and hostname resembling the peer endpoint of the respective TCP connection into the specified field at the end of the nested pipeline.

tls = record (optional)

TLS configuration. Provide an empty record (tls={}) to enable TLS with defaults or set fields to customize it.

{
  skip_peer_verification: bool, // skip certificate verification.
  cacert: string,               // CA bundle to verify peers.
  certfile: string,             // client certificate to present.
  keyfile: string,              // private key for the client certificate.
  min_version: string,          // minimum TLS version (`"1.0"`, `"1.1"`, `"1.2"`, "1.3"`).
  ciphers: string,              // OpenSSL cipher list string.
  client_ca: string,            // CA to validate client certificates.
  require_client_cert,          // require clients to present a certificate.
}

The client_ca and require_client_cert options are only applied for operators that accept incoming client connections, and otherwise ignored.

Any value not specified in the record will either be picked up from the configuration or if not configured will not be used by the operator.

See the Node TLS Setup guide for more details.

tls = record (optional)

TLS configuration. Provide an empty record (tls={}) to enable TLS with defaults or set fields to customize it.

{
  skip_peer_verification: bool, // skip certificate verification.
  cacert: string,               // CA bundle to verify peers.
  certfile: string,             // client certificate to present.
  keyfile: string,              // private key for the client certificate.
  min_version: string,          // minimum TLS version (`"1.0"`, `"1.1"`, `"1.2"`, "1.3"`).
  ciphers: string,              // OpenSSL cipher list string.
  client_ca: string,            // CA to validate client certificates.
  require_client_cert,          // require clients to present a certificate.
}

The client_ca and require_client_cert options are only applied for operators that accept incoming client connections, and otherwise ignored.

Any value not specified in the record will either be picked up from the configuration or if not configured will not be used by the operator.

See the Node TLS Setup guide for more details.

max_buffered_chunks = int (optional)

Maximum number of buffered chunks per connection.

Defaults to 10.

{ … } (optional)

The pipeline to run for each individual TCP connection. If none is specified, no transformations are applied to the output streams. Unless you are sure that there is at most one active connection at a time, it is recommended to specify a pipeline that parses the individual connection streams into events, for instance { read_json }. Otherwise, the output can be interleaved.

Examples

Listen for incoming Syslog over TCP

Listen on all network interfaces, parsing each individual connection as syslog:

load_tcp "0.0.0.0:8090" { read_syslog }

Connect to a remote endpoint and read JSON

// We know that there is only one connection, so we do not specify a pipeline.
load_tcp "example.org:8090", connect=true
read_json

Listen on localhost with TLS enabled

Wait for connections on localhost with TLS enabled, parsing incoming JSON streams according to the schema "my_schema", forwarding no more than 20 events per individual connection:

load_tcp "127.0.0.1:4000", tls={certfile: "key_and_cert.pem", keyfile: "key_and_cert.pem"} {
  read_json schema="my_schema"
  head 20
}

This example may use a self-signed certificate that can be generated like this:

uv run --with trustme python -m trustme

You can test the endpoint locally by issuing a TLS connection:

openssl s_client 127.0.0.1:4000

Listen with mutual TLS (mTLS) authentication

Require clients to present valid certificates signed by a trusted CA:

load_tcp "0.0.0.0:4000", tls={certfile: "server.pem", keyfile: "server-key.pem", tls_client_ca: "ca.pem", tls_require_client_cert: true} {
  read_json
}

Generate test certificates:

# Create CA
openssl req -x509 -new -newkey rsa:2048 -nodes -keyout ca-key.pem -out ca.pem -days 365 -subj "/CN=Test CA"

# Create server certificate
openssl req -new -newkey rsa:2048 -nodes -keyout server-key.pem -out server.csr -subj "/CN=localhost"
openssl x509 -req -in server.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out server.pem -days 365

# Create client certificate
openssl req -new -newkey rsa:2048 -nodes -keyout client-key.pem -out client.csr -subj "/CN=client"
openssl x509 -req -in client.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out client.pem -days 365

Test the mTLS connection:

openssl s_client -connect 127.0.0.1:4000 -cert client.pem -key client-key.pem -CAfile ca.pem