Functions appear in expressions and take positional and/or named arguments, producing a value as a result of their computation.
Function signatures have the following notation:
f(arg1:<type>, arg2=<type>, [arg3=type]) -> <type>
arg:<type>
: positional argumentarg=<type>
: named argument[arg=type]
: optional (named) argument-> <type>
: function return type
TQL features the uniform function call syntax
(UFCS), which
allows you to interchangeably call a function with at least one argument either
as free function or method. For example, length(str)
and str.length()
resolve to the identical function call. The latter syntax is particularly
suitable for function chaining, e.g., x.f().g().h()
reads left-to-right as
“start with x
, apply f()
, then g()
and then h()
,” compared to
h(g(f(x)))
, which reads “inside out.”
Throughout our documentation, we use the free function style in the synopsis but often resort to the method style when it is more idiomatic.
Aggregation
Section titled “Aggregation”all
→all([true,true,false])
any
→any([true,false,true])
count
→count([1,2,null])
count_distinct
→count_distinct([1,2,2,3])
count_if
→count_if([1,2,null], x => x > 1)
distinct
→distinct([1,2,2,3])
first
→first([null,2,3])
last
→last([1,2,null])
max
→max([1,2,3])
mean
→mean([1,2,3])
median
→median([1,2,3,4])
min
→min([1,2,3])
mode
→mode([1,1,2,3])
otherwise
→fallback
value if primary
is null
.x.otherwise(0)
quantile
→quantile([1,2,3,4], q=0.5)
stddev
→stddev([1,2,3])
sum
→sum([1,2,3])
value_counts
→value_counts([1,2,2,3])
variance
→variance([1,2,3])
Bit Operations
Section titled “Bit Operations”bit_and
→bit_and(lhs, rhs)
bit_or
→bit_or(lhs, rhs)
bit_xor
→bit_xor(lhs, rhs)
shift_left
→shift_left(lhs, rhs)
shift_right
→shift_right(lhs, rhs)
Decoding
Section titled “Decoding”decode_base64
→decode_base64("VGVuemly")
decode_hex
→decode_hex("4e6f6E6365")
decode_url
→decode_url("Hello%20World")
Encoding
Section titled “Encoding”encode_base64
→encode_base64("Tenzir")
encode_hex
→encode_hex("Tenzir")
encode_url
→encode_url("Hello World")
Hashing
Section titled “Hashing”hash_md5
→hash_md5("foo")
hash_sha1
→hash_sha1("foo")
hash_sha224
→hash_sha224("foo")
hash_sha256
→hash_sha256("foo")
hash_sha384
→hash_sha384("foo")
hash_sha512
→hash_sha512("foo")
hash_xxh3
→hash_xxh3("foo")
is_v4
→is_v4(1.2.3.4)
network
→10.0.0.0/8.network()
append
→xs.append(y)
concatenate
→concatenate(xs, ys)
get
→list.get(index, default)
length
→[1,2,3].length()
map
→xs.map(x => x + 3)
prepend
→xs.prepend(y)
sort
→xs.sort()
where
→xs.where(x, x > 5)
zip
→zip(xs, ys)
abs
→abs(-42)
Networking
Section titled “Networking”community_id
→community_id(src_ip=1.2.3.4, dst_ip=4.5.6.7, proto="tcp")
encrypt_cryptopan
→encrypt_cryptopan(1.2.3.4)
Parsing
Section titled “Parsing”parse_cef
→string.parse_cef()
parse_csv
→string.parse_csv(header=["a","b"])
parse_grok
→string.parse_grok("%{IP:client} …")
parse_json
→string.parse_json()
parse_kv
→string.parse_kv()
parse_leef
→string.parse_leef()
parse_ssv
→string.parse_ssv(header=["a","b"])
parse_syslog
→string.parse_syslog()
parse_tsv
→string.parse_tsv(header=["a","b"])
parse_xsv
→string.parse_xsv(",", ";", "", header=["a","b"])
parse_yaml
→string.parse_yaml()
Printing
Section titled “Printing”print_csv
→record.print_csv()
print_json
→record.print_json()
print_kv
→record.print_kv()
print_ndjson
→record.print_ndjson()
print_ssv
→record.print_ssv()
print_tsv
→record.print_tsv()
print_xsv
→record.print_tsv()
print_yaml
→record.print_yaml()
Record
Section titled “Record”get
→list.get(index, default)
has
→record.has("field")
keys
→record.keys()
sort
→xs.sort()
Runtime
Section titled “Runtime”env
→env("PATH")
secret
→secret("KEY")
Subnet
Section titled “Subnet”network
→10.0.0.0/8.network()
Time & Date
Section titled “Time & Date”count_days
→days
in a duration.count_days(100d)
count_hours
→hours
in a duration.count_hours(100d)
count_microseconds
→microseconds
in a duration.count_microseconds(100d)
count_milliseconds
→milliseconds
in a duration.count_milliseconds(100d)
count_minutes
→minutes
in a duration.count_minutes(100d)
count_months
→months
in a duration.count_months(100d)
count_nanoseconds
→nanoseconds
in a duration.count_nanoseconds(100d)
count_seconds
→seconds
in a duration.count_seconds(100d)
count_weeks
→weeks
in a duration.count_weeks(100d)
count_years
→years
in a duration.count_years(100d)
format_time
→ts.format_time("%d/ %m/%Y")
from_epoch
→from_epoch(time_ms * 1ms)
microseconds
→microseconds(100)
milliseconds
→milliseconds(100)
minutes
→minutes(100)
months
→months(100)
nanoseconds
→nanoseconds(100)
parse_time
→"10/11/2012".parse_time("%d/%m/%Y")
seconds
→seconds(100)
since_epoch
→since_epoch(2021-02-24)
String
Section titled “String”Filesystem
Section titled “Filesystem”file_contents
→file_contents("/path/to/file")
file_name
→file_name("/path/to/log.json")
parent_dir
→parent_dir("/path/to/log.json")
Inspection
Section titled “Inspection”ends_with
→"hello".ends_with("lo")
is_alnum
→"hello123".is_alnum()
is_alpha
→"hello".is_alpha()
is_lower
→"hello".is_lower()
is_numeric
→"1234".is_numeric()
is_printable
→"hello".is_printable()
is_title
→"Hello World".is_title()
is_upper
→"HELLO".is_upper()
length_bytes
→"hello".length_bytes()
length_chars
→"hello".length_chars()
match_regex
→"Hi".match_regex("[Hh]i")
slice
→"Hi".slice(begin=2, stride=4)
starts_with
→"hello".starts_with("he")
Transformation
Section titled “Transformation”capitalize
→"hello".capitalize()
join
→join(["a", "b", "c"], ",")
replace
→"hello".replace("o", "a")
replace_regex
→"hello".replace("l+o", "y")
reverse
→"hello".reverse()
split
→split("a,b,c", ",")
split_regex
→split_regex("a1b2c", r"\d")
to_lower
→"HELLO".to_lower()
to_title
→"hello world".to_title()
to_upper
→"hello".to_upper()
trim
→" hello ".trim()
trim_end
→"hello ".trim_end()
trim_start
→" hello".trim_start()
Type System
Section titled “Type System”Conversion
Section titled “Conversion”duration
→duration("1.34w")
float
→float(42)
int
→int(-4.2)
ip
→ip("1.2.3.4")
string
→string(1.2.3.4)
subnet
→subnet("1.2.3.4/16")
time
→time("2020-03-15")
Introspection
Section titled “Introspection”type_id
→type_id(1 + 3.2)
Transposition
Section titled “Transposition”flatten
→flatten(this)
unflatten
→unflatten(this)