Add operators

This guide shows you how to create user-defined operators (UDOs) for your package. You’ll learn how to define operators with positional and named arguments, and how to test them with the Test Framework.

Create a user-defined operator

User-defined operators (UDOs) are reusable building blocks that you can use in your pipelines. Place operator files in the operators directory of your package.

Tenzir names operators using the convention <package>::[dirs...]::<basename>. For example, a file at operators/ocsf/map.tql in a package with ID acme becomes the operator acme::ocsf::map.

operators/ocsf/auth.tql

// Normalize authentication logs to OCSF Authentication event class
class_uid = 3002
category_uid = 3
activity_id = 1 if outcome == "success" else 2
actor.user.name = username
actor.user.uid = user_id
src_endpoint.ip = src_ip
dst_endpoint.ip = dst_ip
drop username, user_id, src_ip, dst_ip, outcome

After installing the package, use the operator in any pipeline:

from_file "auth.json"
acme::ocsf::auth
publish "ocsf-events"

Add parameters to operators

Operators can accept positional and named arguments, enabling you to create flexible, reusable building blocks that match the calling conventions of built-in operators. Define parameters in a YAML frontmatter block at the beginning of the .tql file.

Parameter schema

Each parameter supports the following fields:

Field	Required	Description
name	Yes	Parameter name, used as $name in the operator body
type	No	Type constraint for the parameter value
description	No	Documentation string for the parameter
default	No	Default value if the argument is not provided

Supported types

The type field constrains what values the parameter accepts. It uses TQL type definition syntax:

Type	Description
field	A field selector (for example, name). Cannot have non-null defaults
string	A string literal or expression
int	A signed integer value
uint	An unsigned integer value
float	A floating-point value
bool	A boolean value
duration	A duration value
time	A timestamp value
ip	An IP address
subnet	A subnet value
blob	A blob value
secret	A secret string (accepts string literals)

If you omit the type field, the parameter accepts any value.

Null values

All typed parameters accept null values regardless of their declared type.

Type checking

Tenzir validates parameter types at compile time when possible:

• Compile-time checking occurs when arguments are constant values
• Runtime checking defers validation for expressions containing runtime data

If a type mismatch occurs, Tenzir reports an error with the expected type and shows usage information for the operator.

Work with operator parameters

This section shows common patterns for defining and using operator parameters.

Define positional arguments

Positional arguments are passed in order when calling the operator. Define them under the args.positional key:

operators/tag.tql

---
args:
  positional:
    - name: field
      type: field
    - name: value
      type: string
---

$field = $value

Call this operator with positional arguments:

from {x: 1}
acme::tag name, "Alice"

{x: 1, name: "Alice"}

Define named arguments

Named arguments use the name=value syntax and can have default values. Define them under the args.named key:

operators/tag.tql

---
args:
  positional:
    - name: field
      type: field
    - name: value
      type: string
  named:
    - name: prefix
      type: string
      default: ""
---

$field = f"{$prefix}{$value}"

Call this operator with both positional and named arguments:

from {x: 1}
acme::tag name, "Alice", prefix="User: "

{x: 1, name: "User: Alice"}

Make arguments optional

Positional arguments with a default value become optional. Callers can omit them, and Tenzir substitutes the default:

operators/greet.tql

---
args:
  positional:
    - name: name
      type: string
      default: "World"
---

greeting = f"Hello, {$name}!"

Calling the operator without arguments uses the default value:

from {}
acme::greet

{greeting: "Hello, World!"}

Passing an explicit argument overrides the default:

from {}
acme::greet "Alice"

{greeting: "Hello, Alice!"}

Use field parameters

The field type enables dynamic field selection. The caller passes a field path, and the operator uses it to read or write data:

operators/double_value.tql

---
args:
  positional:
    - name: target
      type: field
---

$target = $target * 2

Using the operator:

from {count: 5, score: 10}
acme::double_value count

{count: 10, score: 10}

Detect whether an argument was provided

Any typed parameter supports default: null, making it optional without requiring a concrete fallback value. Inside the operator body, compare the parameter against null to check whether the caller provided it.

This is especially useful for field parameters, which cannot have non-null defaults because a field selector is not a constant value:

operators/redact.tql

---
args:
  positional:
    - name: input
      type: field
  named:
    - name: target
      type: field
      default: null
      description: "Optional field to redact"
---

if $target != null {
  $target = "REDACTED"
}
$input = $input

When the caller omits target, the operator skips the redaction:

from {message: "hello", secret: "s3cret"}
acme::redact message

{message: "hello", secret: "s3cret"}

When the caller provides target, the operator redacts that field:

from {message: "hello", secret: "s3cret"}
acme::redact message, target=secret

{message: "hello", secret: "REDACTED"}

The same pattern works for other types. For example, an optional string parameter that only applies when provided:

operators/tag.tql

---
args:
  positional:
    - name: field
      type: field
  named:
    - name: suffix
      type: string
      default: null
---

if $suffix != null {
  $field = f"{$field}{$suffix}"
}

Note

Callers cannot pass null explicitly for a field parameter. Writing target=null produces a compile-time error because null is not a valid field selector. The null default applies only when the argument is omitted entirely. Other parameter types do accept explicit null values.

Call other operators

Parameterized operators can call other operators, including passing through their own parameters:

operators/transform.tql

---
args:
  positional:
    - name: field
      type: field
  named:
    - name: multiplier
      type: int
      default: 2
---

utils::scale $field, factor=$multiplier

Modularize packages with operator hierarchies

Complex packages benefit from breaking functionality into a hierarchy of operators that call each other. This pattern keeps individual operators focused, enables independent testing, and creates a clear structure that mirrors your directory layout.

Consider a package that normalizes various event types to OCSF. Keep source-specific cleanup, shared OCSF setup, and dispatch in one main mapper, then delegate only event-specific mapping details to leaf operators:

Operator	Purpose
acme::ocsf::map	Main mapper: cleanup, shared OCSF setup, and dispatch
acme::ocsf::base	Fallback: maps to OCSF Base Event
acme::ocsf::events::*	Event-specific mapping such as dns, http, or auth

This hierarchy maps directly to your directory structure:

• Directoryoperators/

  • Directoryocsf/

    • base.tql
    • map.tql
    • Directoryevents/

      • dns.tql
      • http.tql
      • auth.tql

The main mapper keeps source residue in a product-specific acme namespace, performs source-specific cleanup, adds shared OCSF fields, routes events to specialized mappers based on a stable event discriminator, and returns the mapped OCSF event:

operators/ocsf/map.tql

this = { acme: this }

ocsf.metadata = {
  product: {
    name: "ACME Logs",
    vendor_name: "ACME",
  },
  version: "1.8.0",
}
ocsf.severity_id = 1

match acme.event_type {
  "dns" => {
    acme::ocsf::events::dns
  }
  "http" => {
    acme::ocsf::events::http
  }
  "auth" => {
    acme::ocsf::events::auth
  }
  _ => {
    acme::ocsf::base
  }
}

this = {...ocsf, unmapped: acme}

The fallback operator ensures that unrecognized events still conform to OCSF by mapping them to the Base Event class:

operators/ocsf/base.tql

@name = "ocsf.base_event"
ocsf.category_uid = 0
ocsf.class_uid = 0
ocsf.activity_id = 0
ocsf.type_uid = 0
ocsf.severity_id = 0
ocsf.time = now()

Each leaf operator handles one specific event type:

operators/ocsf/events/dns.tql

@name = "ocsf.dns_activity"
ocsf.category_uid = 4
ocsf.class_uid = 4003
ocsf.activity_id = 1
ocsf.type_uid = ocsf.class_uid * 100 + ocsf.activity_id

ocsf.query.hostname = move acme.query_name
ocsf.query.type = move acme.query_type
ocsf.answers = move acme.dns_answers
// ... additional field mappings

After the package mapper runs, callers can run the shared OCSF helpers. The mapper should produce minimal OCSF, and ocsf::derive expands it with derived sibling fields before ocsf::cast validates the final shape:

acme::ocsf::map
ocsf::derive
ocsf::cast

These operators compose into an end-to-end pipeline:

from_file "logs/mixed.json" {
  read_json
}
acme::ocsf::map
ocsf::derive
ocsf::cast
publish "ocsf"

The parser stays with the source operator, while acme::ocsf::map owns cleanup, shared OCSF setup, and dispatch.

Design principles

When building operator hierarchies, follow these guidelines:

• Expose one mapper contract: Accept parsed source events through a main package mapper, for example acme::ocsf::map.
• Keep cleanup close to mapping: Put source-specific normalization and shared OCSF setup in the main mapper before dispatch.
• Produce minimal OCSF: Set identifiers and source-derived attributes in the mapper, then use ocsf::derive to add derived sibling fields.
• Use dispatchers for routing: Route events based on type or other criteria.
• Mirror directory structure: Operator names reflect their location.
• Provide fallbacks: Handle unrecognized inputs gracefully.

See also

• Create a package
• Add pipelines
• Test packages
• Write a package