Functions appear in expressions and take positional and/or named arguments, producing a value as a result of their computation.
Function signatures have the following notation:
f(arg1:<type>, arg2=<type>, [arg3=type]) -> <type>arg:<type>: positional argumentarg=<type>: named argument[arg=type]: optional (named) argument-> <type>: function return type
TQL features the uniform function call syntax
(UFCS), which
allows you to interchangeably call a function with at least one argument either
as free function or method. For example, length(str) and str.length()
resolve to the identical function call. The latter syntax is particularly
suitable for function chaining, e.g., x.f().g().h() reads left-to-right as
“start with x, apply f(), then g() and then h(),” compared to
h(g(f(x))), which reads “inside out.”
Throughout our documentation, we use the free function style in the synopsis but often resort to the method style when it is more idiomatic.
Aggregation
Section titled “Aggregation”all
→Computes the conjunction (AND) of all grouped boolean values.
all([true,true,false])any
→Computes the disjunction (OR) of all grouped boolean values.
any([true,false,true])count
→Counts the events or non-null grouped values.
count([1,2,null])count_distinct
→Counts all distinct non-null grouped values.
count_distinct([1,2,2,3])count_if
→Counts the events or non-null grouped values matching a given predicate.
count_if([1,2,null], x => x > 1)distinct
→Creates a sorted list without duplicates of non-null grouped values.
distinct([1,2,2,3])first
→Takes the first non-null grouped value.
first([null,2,3])last
→Takes the last non-null grouped value.
last([1,2,null])max
→Computes the maximum of all grouped values.
max([1,2,3])mean
→Computes the mean of all grouped values.
mean([1,2,3])median
→Computes the approximate median of all grouped values using a t-digest algorithm.
median([1,2,3,4])min
→Computes the minimum of all grouped values.
min([1,2,3])mode
→Takes the most common non-null grouped value.
mode([1,1,2,3])otherwise
→Returns a
fallback value if primary is null.x.otherwise(0)quantile
→Computes the specified quantile of all grouped values.
quantile([1,2,3,4], q=0.5)stddev
→Computes the standard deviation of all grouped values.
stddev([1,2,3])sum
→Computes the sum of all values.
sum([1,2,3])value_counts
→Returns a list of all grouped values alongside their frequency.
value_counts([1,2,2,3])variance
→Computes the variance of all grouped values.
variance([1,2,3])Bit Operations
Section titled “Bit Operations”bit_and
→Computes the bit-wise AND of its arguments.
bit_and(lhs, rhs)bit_or
→Computes the bit-wise OR of its arguments.
bit_or(lhs, rhs)bit_xor
→Computes the bit-wise XOR of its arguments.
bit_xor(lhs, rhs)shift_left
→Performs a bit-wise left shift.
shift_left(lhs, rhs)shift_right
→Performs a bit-wise right shift.
shift_right(lhs, rhs)Decoding
Section titled “Decoding”decode_base64
→Decodes bytes as Base64.
decode_base64("VGVuemly")decode_hex
→Decodes bytes from their hexadecimal representation.
decode_hex("4e6f6E6365")decode_url
→Decodes URL encoded strings.
decode_url("Hello%20World")Encoding
Section titled “Encoding”encode_base64
→Encodes bytes as Base64.
encode_base64("Tenzir")encode_hex
→Encodes bytes into their hexadecimal representation.
encode_hex("Tenzir")encode_url
→Encodes strings using URL encoding.
encode_url("Hello World")Hashing
Section titled “Hashing”hash_md5
→Computes an MD5 hash digest.
hash_md5("foo")hash_sha1
→Computes a SHA-1 hash digest.
hash_sha1("foo")hash_sha224
→Computes a SHA-224 hash digest.
hash_sha224("foo")hash_sha256
→Computes a SHA-256 hash digest.
hash_sha256("foo")hash_sha384
→Computes a SHA-384 hash digest.
hash_sha384("foo")hash_sha512
→Computes a SHA-512 hash digest.
hash_sha512("foo")hash_xxh3
→Computes an XXH3 hash digest.
hash_xxh3("foo")is_v4
→Checks whether an IP address has version number 4.
is_v4(1.2.3.4)network
→Retrieves the network address of a subnet.
10.0.0.0/8.network()append
→Inserts an element at the back of a list.
xs.append(y)concatenate
→Merges two lists.
concatenate(xs, ys)get
→Gets a field from a record or an element from a list
list.get(index, default)length
→Retrieves the length of a list.
[1,2,3].length()map
→Maps each list element to an expression.
xs.map(x => x + 3)prepend
→Inserts an element at the start of a list.
xs.prepend(y)sort
→Sorts lists and record fields.
xs.sort()where
→Filters list elements based on a predicate.
xs.where(x, x > 5)zip
→Combines two lists into a list of pairs.
zip(xs, ys)abs
→Returns the absolute value.
abs(-42)Networking
Section titled “Networking”community_id
→Computes the Community ID for a network connection/flow.
community_id(src_ip=1.2.3.4, dst_ip=4.5.6.7, proto="tcp")encrypt_cryptopan
→Encrypts an IP address via Crypto-PAn.
encrypt_cryptopan(1.2.3.4)Parsing
Section titled “Parsing”parse_cef
→Parses a string as a CEF message
string.parse_cef()parse_csv
→Parses a string as CSV (Comma-Separated Values).
string.parse_csv(header=["a","b"])parse_grok
→Parses a string according to a grok pattern.
string.parse_grok("%{IP:client} …")parse_json
→Parses a string as a JSON value.
string.parse_json()parse_kv
→Parses a string as key-value pairs.
string.parse_kv()parse_leef
→Parses a string as a LEEF message
string.parse_leef()parse_ssv
→Parses a string as space separated values.
string.parse_ssv(header=["a","b"])parse_syslog
→Parses a string as a Syslog message.
string.parse_syslog()parse_tsv
→Parses a string as tab separated values.
string.parse_tsv(header=["a","b"])parse_xsv
→Parses a string as delimiter separated values.
string.parse_xsv(",", ";", "", header=["a","b"])parse_yaml
→Parses a string as a YAML value.
string.parse_yaml()Printing
Section titled “Printing”print_csv
→Prints a record as a comma-separated string of values.
record.print_csv()print_json
→Transforms a value into a JSON string.
record.print_json()print_kv
→Prints records in a key-value format.
record.print_kv()print_ndjson
→Transforms a value into a single-line JSON string.
record.print_ndjson()print_ssv
→Prints a record as a space-separated string of values.
record.print_ssv()print_tsv
→Prints a record as a tab-separated string of values.
record.print_tsv()print_xsv
→Prints a record as a delimited sequence of values.
record.print_tsv()print_yaml
→Prints a value as a YAML document.
record.print_yaml()Record
Section titled “Record”get
→Gets a field from a record or an element from a list
list.get(index, default)has
→Checks whether a record has a specified field.
record.has("field")keys
→Retrieves a list of field names from a record.
record.keys()sort
→Sorts lists and record fields.
xs.sort()Runtime
Section titled “Runtime”env
→Reads an environment variable.
env("PATH")secret
→Reads a secret from a store.
secret("KEY")Subnet
Section titled “Subnet”network
→Retrieves the network address of a subnet.
10.0.0.0/8.network()Time & Date
Section titled “Time & Date”count_days
→Counts the number of
days in a duration.count_days(100d)count_hours
→Counts the number of
hours in a duration.count_hours(100d)count_microseconds
→Counts the number of
microseconds in a duration.count_microseconds(100d)count_milliseconds
→Counts the number of
milliseconds in a duration.count_milliseconds(100d)count_minutes
→Counts the number of
minutes in a duration.count_minutes(100d)count_months
→Counts the number of
months in a duration.count_months(100d)count_nanoseconds
→Counts the number of
nanoseconds in a duration.count_nanoseconds(100d)count_seconds
→Counts the number of
seconds in a duration.count_seconds(100d)count_weeks
→Counts the number of
weeks in a duration.count_weeks(100d)count_years
→Counts the number of
years in a duration.count_years(100d)format_time
→Formats a time into a string that follows a specific format.
ts.format_time("%d/ %m/%Y")from_epoch
→Interprets a duration as Unix time.
from_epoch(time_ms * 1ms)microseconds
→Converts a number to equivalent microseconds.
microseconds(100)milliseconds
→Converts a number to equivalent milliseconds.
milliseconds(100)minutes
→Converts a number to equivalent minutes.
minutes(100)months
→Converts a number to equivalent months.
months(100)nanoseconds
→Converts a number to equivalent nanoseconds.
nanoseconds(100)parse_time
→Parses a time from a string that follows a specific format.
"10/11/2012".parse_time("%d/%m/%Y")seconds
→Converts a number to equivalent seconds.
seconds(100)since_epoch
→Interprets a time value as duration since the Unix epoch.
since_epoch(2021-02-24)String
Section titled “String”Filesystem
Section titled “Filesystem”file_contents
→Reads a file's contents.
file_contents("/path/to/file")file_name
→Extracts the file name from a file path.
file_name("/path/to/log.json")parent_dir
→Extracts the parent directory from a file path.
parent_dir("/path/to/log.json")Inspection
Section titled “Inspection”ends_with
→Checks if a string ends with a specified substring.
"hello".ends_with("lo")is_alnum
→Checks if a string is alphanumeric.
"hello123".is_alnum()is_alpha
→Checks if a string contains only alphabetic characters.
"hello".is_alpha()is_lower
→Checks if a string is in lowercase.
"hello".is_lower()is_numeric
→Checks if a string contains only numeric characters.
"1234".is_numeric()is_printable
→Checks if a string contains only printable characters.
"hello".is_printable()is_title
→Checks if a string follows title case.
"Hello World".is_title()is_upper
→Checks if a string is in uppercase.
"HELLO".is_upper()length_bytes
→Returns the length of a string in bytes.
"hello".length_bytes()length_chars
→Returns the length of a string in characters.
"hello".length_chars()match_regex
→Checks if a string partially matches a regular expression.
"Hi".match_regex("[Hh]i")slice
→Slices a string with offsets and strides.
"Hi".slice(begin=2, stride=4)starts_with
→Checks if a string starts with a specified substring.
"hello".starts_with("he")Transformation
Section titled “Transformation”capitalize
→Capitalizes the first character of a string.
"hello".capitalize()join
→Joins a list of strings into a single string using a separator.
join(["a", "b", "c"], ",")replace
→Replaces characters within a string.
"hello".replace("o", "a")replace_regex
→Replaces characters within a string based on a regular expression.
"hello".replace("l+o", "y")reverse
→Reverses the characters of a string.
"hello".reverse()split
→Splits a string into substrings.
split("a,b,c", ",")split_regex
→Splits a string into substrings with a regex.
split_regex("a1b2c", r"\d")to_lower
→Converts a string to lowercase.
"HELLO".to_lower()to_title
→Converts a string to title case.
"hello world".to_title()to_upper
→Converts a string to uppercase.
"hello".to_upper()trim
→Trims whitespace from both ends of a string.
" hello ".trim()trim_end
→Trims whitespace from the end of a string.
"hello ".trim_end()trim_start
→Trims whitespace from the start of a string.
" hello".trim_start()Type System
Section titled “Type System”Conversion
Section titled “Conversion”duration
→Casts an expression to a duration value.
duration("1.34w")float
→Casts an expression to a float.
float(42)int
→Casts an expression to an integer.
int(-4.2)ip
→Casts an expression to an IP address.
ip("1.2.3.4")string
→Casts an expression to a string.
string(1.2.3.4)subnet
→Casts an expression to a subnet value.
subnet("1.2.3.4/16")time
→Casts an expression to a time value.
time("2020-03-15")Introspection
Section titled “Introspection”type_id
→Retrieves the type id of an expression.
type_id(1 + 3.2)Transposition
Section titled “Transposition”flatten
→Flattens nested data.
flatten(this)unflatten
→Unflattens nested data.
unflatten(this)