Functions

Functions appear in expressions and take positional and/or named arguments, producing a value as a result of their computation.

Function signatures have the following notation:

f(arg1:<type>, arg2=<type>, [arg3=type]) -> <type>

• arg:<type>: positional argument
• arg=<type>: named argument
• [arg=type]: optional (named) argument
• -> <type>: function return type

TQL features the uniform function call syntax (UFCS), which allows you to interchangeably call a function with at least one argument either as free function or method. For example, length(str) and str.length() resolve to the identical function call. The latter syntax is particularly suitable for function chaining, e.g., x.f().g().h() reads left-to-right as "start with x, apply f(), then g() and then h()," compared to h(g(f(x))), which reads "inside out."

Throughout our documentation, we use the free function style in the synopsis but often resort to the method style when it is more idiomatic.

Aggregation

Function	Description	Example
all	Computes the conjunction (AND) of all boolean values	all([true,true,false])
any	Computes the disjunction (OR) of all boolean values	any([true,false,true])
collect	Creates a list of all non-null values, preserving duplicates	collect([1,2,2,3])
count	Counts the events or non-null values	count([1,2,null])
count_distinct	Counts all distinct non-null values	count_distinct([1,2,2,3])
distinct	Creates a sorted list without duplicates of non-null values	distinct([1,2,2,3])
first	Takes the first non-null value	first([null,2,3])
last	Takes the last non-null value	last([1,2,null])
max	Computes the maximum of all values	max([1,2,3])
mean	Computes the mean of all values	mean([1,2,3])
median	Computes the approximate median with a t-digest algorithm	median([1,2,3,4])
min	Computes the minimum of all values	min([1,2,3])
mode	Takes the most common non-null value	mode([1,1,2,3])
quantile	Computes the specified quantile q of values	quantile([1,2,3,4], q=0.5)
stddev	Computes the standard deviation of all values	stddev([1,2,3])
sum	Computes the sum of all values	sum([1,2,3])
value_counts	Returns a list of values with their frequency	value_counts([1,2,2,3])
variance	Computes the variance of all values	variance([1,2,3])

List

Function	Description	Example
append	Inserts an element at the back of a list	xs.append(y)
prepend	Inserts an element at the front of a list	xs.prepend(y)
concatenate	Merges two lists	concatenate(xs, ys)
map	Maps each list element to an expression	xs.map(x, x + 3)
where	Filters list elements based on a predicate	xs.where(x, x > 5)
zip	Combines two lists into a list of pairs	zip(xs, ys)

String

Inspection

Function	Description	Example
length_bytes	Returns the length of a string in bytes	"hello".length_bytes()
length_chars	Returns the length of a string in characters	"hello".length_chars()
starts_with	Checks if a string starts with a substring	"hello".starts_with("he")
ends_with	Checks if a string ends with a substring	"hello".ends_with("lo")
is_alnum	Checks if a string is alphanumeric	"hello123".is_alnum()
is_alpha	Checks if a string contains only letters	"hello".is_alpha()
is_lower	Checks if a string is in lowercase	"hello".is_lower()
is_numeric	Checks if a string contains only numbers	"1234".is_numeric()
is_printable	Checks if a string contains only printable characters	"hello".is_printable()
is_title	Checks if a string follows title case	"Hello World".is_title()
is_upper	Checks if a string is in uppercase	"HELLO".is_upper()

Transformation

Function	Description	Example
trim	Trims whitespace from both ends of a string	" hello ".trim()
trim_start	Trims whitespace from the start of a string	" hello".trim_start()
trim_end	Trims whitespace from the end of a string	"hello ".trim_end()
capitalize	Capitalizes the first character of a string	"hello".capitalize()
replace	Replaces characters within a string	"hello".replace("o", "a")
replace_regex	Reverses the characters of a string	"hello".replace("l+o", "y")
reverse	Reverses the characters of a string	"hello".reverse()
to_lower	Converts a string to lowercase	"HELLO".to_lower()
to_title	Converts a string to title case	"hello world".to_title()
to_upper	Converts a string to uppercase	"hello".to_upper()
split	Splits a string into substrings	split("a,b,c", ",")
split_regex	Splits a string into substrings with a regex	split_regex("a1b2c", r"\d")
join	Joins a list of strings into a single string	join(["a", "b", "c"], ",")

File Paths

Function	Description	Example
file_name	Extracts the file name from a file path	file_name("/path/to/log.json")
parent_dir	Extracts the parent directory from a file path	parent_dir("/path/to/log.json")

Time & Date

Function	Description	Example
as_secs	Converts a duration into seconds	as_secs(42ms)
from_epoch	Interprets a duration as Unix time	from_epoch(time_ms * 1ms)
now	Gets the current wallclock time	now()
since_epoch	Turns a time into a duration since the Unix epoch	since_epoch(2021-02-24)
parse_time	Parses a timestamp following a given format	"10/11/2012".parse_time("%d/%m/%Y")

Math

Function	Description	Example
ceil	Takes the ceiling	ceil(4.2), ceil(3.2s, 1m)
floor	Takes the floor	floor(4.2), floor(32h, 1d)
random	Generates a random number	random()
round	Rounds a value	round(4.2), round(31m, 1h)
sqrt	Calculates the square root	sqrt(49)

Networking

Function	Description	Example
community_id	Computes a Community ID	community_id(src_ip=1.2.3.4, dst_ip=4.5.6.7, proto="tcp")
decapsulate	Decapsulates PCAP packets	decapsulate(this)
encrypt_cryptopan	Encrypts IPs via Crypto-PAn	encrypt_cryptopan(1.2.3.4)
is_v4	Checks if an IP is IPv4	is_v4(1.2.3.4)
is_v6	Checks if an IP is IPv6	is_v6(::1)

Hashing

Function	Description	Example
hash_md5	Computes a MD5 hash digest	hash_md5("foo")
hash_sha1	Computes a SHA1 hash digest	hash_sha1("foo")
hash_sha224	Computes a SHA224 hash digest	hash_sha224("foo")
hash_sha256	Computes a SHA256 hash digest	hash_sha256("foo")
hash_sha384	Computes a SHA384 hash digest	hash_sha384("foo")
hash_sha512	Computes a SHA512 hash digest	hash_sha512("foo")
hash_xxh3	Computes a XXH3 hash digest	hash_xxh3("foo")

Encoding

function	description	example
encode_base64	Encodes bytes as Base64	encode_base64("Tenzir")
encode_hex	Encodes bytes as their hexadecimal representation	encode_hex("Tenzir")

Decoding

function	description	example
decode_base64	Decodes bytes as Base64	decode_base64("VGVuemly")
decode_hex	Decodes bytes from their hexadecimal representation	decode_hex("4e6f6E6365")

Type System

Introspection

Function	Description	Example
type_id	Retrieves the type of an expression	type_id(1 + 3.2)
has	Checks whether a record has a field	record.has("field")
length	Retrieves the length of a list	[1,2,3].length()
network	Retrieves the network address of a subnet	10.0.0.0/8.network()

Conversion

Function	Description	Example
int	Casts an expression to a signed integer	int(-4.2)
uint	Casts an expression to an unsigned integer	uint(4.2)
float	Casts an expression to a float	float(42)
time	Casts an expression to a time value	time("2020-03-15")
string	Casts an expression to string	string(1.2.3.4)
ip	Casts an expression to an IP	ip("1.2.3.4")
subnet	Casts an expression to a subnet	subnet("1.2.3.4/16")

Transposition

Function	Description	Example
flatten	Flattens nested data	flatten(this)
unflatten	Unflattens nested structures	unflatten(this)

Runtime

Function	Description	Example
env	Reads an environment variable	env("PATH")
secret	Reads a secret from a store	secret("PATH")