Expressions Tenzir's expression language makes it easy to describe a relevant subset of
interest over structured data. The "easy" part is that Tenzir expressions
operate on multiple different schemas at once, as opposed to traditional
expressions that apply to a single, fixed schema. The language captures this
heterogeneity with extractors .
Expressions occur in pipeline operators. The
where
operator is the most prominent
example.
An expression is a function over an event that evaluates to true
or
false
, indicating whether it qualifies as result. Expression operands are
either sub-expressions or predicates, and can be composed via conjunctions
(and
), disjunctions (or
), and negations (not
).
The following diagram shows an example expression in tree form:
eyJ2ZXJzaW9uIjoiMSIsImVuY29kaW5nIjoiYnN0cmluZyIsImNvbXByZXNzZWQiOnRydWUsImVuY29kZWQiOiJ4nO1d61fiSFx1MDAxNv8+f0VcdTAwMWb3y+6eIV3vx5ydXHUwMDBmoI3KQ3wgqDt7PIFcdTAwMDRcYoaAXHUwMDEwXs6Z/30r6EhcYiRcdTAwMDY1XHUwMDEyu5s+x9ZUhapU7q9+99a9devPX7582XPnXHUwMDAzc++3L3vmrKnbljHUp3u/etcn5nBk9Vx1MDAxZFWEXHUwMDE2f4/642FzUbPjuoPRb1+/6oOBtrxLa/Z7j3eattkzXHUwMDFkd6Tq/lf9/eXLn4uf/rZs21x1MDAxYYzMRfVFwbIxzIJcdTAwMTdP+s6iXYRcdTAwMDGhTEL6XFzBXHUwMDFhXHUwMDFkqLZcXNNQpS3dXHUwMDFlmctcdTAwMTLv0l6+xG5cdTAwMWWKLHeXN29usWR2k47qy0Zblm1fuHN70aVRXz3GsmzkXHUwMDBl+3dm3TLcjiqFgethd1xy++N2xzFHo5V7+lx1MDAwM71puXPvXHUwMDFhXHUwMDAwz1d1p734juWV2eLxl38v7lx1MDAxMCDQ9n7f7lx1MDAwZr22/1x1MDAwMVx1MDAxNp9l61xyvXnXVl1wjGVcdTAwMWTBhCnZss706Yl8zXRMq91xV6+NzMWgUkFcdTAwMTFSXHUwMDAzJ59cdTAwMGK871x1MDAxZlx1MDAxY1x1MDAxYot3+7/lSFx1MDAwZfWeeezd4Yxt2z9cdTAwMWOO8TRcdTAwMWMrXHUwMDA1XHKv4FtQTPyislwiLq45c59cdTAwMWbB935cdTAwMGZcdTAwMDacd2S2Xu7Nvk1l7rh1fTU72Xuu99fTb8t+jlx1MDAwN4b+KC6QXHUwMDAzSTmlnDC4lDjbcu6CfbX7zbulhC2u/vXrZrFe6adPplx0XHUwMDBmk2khMFx1MDAwNlx1MDAwMnJcdTAwMTlbpjc/82aZ7ujNznhopkCqOdU4XCJIMspcYlx1MDAwMT6Me/cjXHUwMDAwNCpcdTAwMTj/u5SKt4i8O9Sd0UBcdTAwMWYq0VpcdTAwMTd7JDSI6LroQ6GRoPBDXHUwMDA0MMJcYonEpT9SQCVhYlx1MDAwYlx1MDAwMV32qu+4XHUwMDE31oP3MnxcdTAwMTLuXc3rPcv2XHUwMDA2XHUwMDFlP19eSO5v3uvzvXfvYta22p5cYu81VWfN4Yp0u5aa+58r9CzDsH2y1lQt6ZZjXHUwMDBlj+PMxf2h1bZcdTAwMWPdrm7qiHpq8+j5VWk+XHUwMDA2aOgj0yv1rpNIbEZQXHUwMDBlpcGLf8NcdTAwMTNcIiWbXFxcblx1MDAxMJ9zro9cdTAwMGWMg/2j+fG8M5lPzGalcX15lW7OQWCVczDYJedcYkEpwUIs58xcdTAwMGblnMe3OD9+ODxs3teqY0Pkj5pcdTAwMGbVkXjSbp4rrkz66eNcdTAwMWOGQ4WaXHUwMDAzLCBcdTAwMTZcYsZcdTAwMTbq6OFIKeko7GpcdTAwMDTCJa3wVSlHq6zD3lwi85Gko8iFx+VcdTAwMWOhpntcbjhMWvij5FMg4Vx1MDAxYqukKac/TIhxXpiJg4zj70fChMNI8OIzNqEgkmG8hUJ47Vx1MDAxNK4n+uioIy6LhVx1MDAwN3po91xySNJNOFx1MDAxNKWJcCBcdTAwMDNY8Vxyljuyclx1MDAxZV9jbtB1KrXs+X51X1x1MDAxNzfzXFz/bt4mn4pxeKjlXHUwMDBlXHRAXHUwMDAyXHUwMDAyuoXpXHUwMDFlPVx1MDAxYyllXHUwMDFjiqPMnI8jnK2sXHUwMDFjwiFcIlSJ/04phyoj+MMox+m7SXFO9GRcdTAwMWPknJWOJEw6MlQhlEpYMIE4vj7Yb9GG6bRcdTAwMGLZUplcdTAwMTlcdTAwMDedfL3XZsV0c1x1MDAwZVxmXHUwMDE4OeSNnNPSXHUwMDFiXHUwMDAw0Fdzjlx1MDAwMFx1MDAwNGP8XHUwMDAxil5cdTAwMTTnXHUwMDEwXHUwMDA0z9r54VxmXHUwMDE4Mlx1MDAwYme1ViZbvoWfinMgXG5dWlOqNJdqjDmOLdbR45FS0lFcdTAwMTaORldFXHUwMDFiJriAtoFWXHUwMDEwXlx1MDAxM2/COFac4ls32cXS2ZZcdTAwMTK4kVSkJlc+eKVWXHUwMDA0yfzn94Q45oXJN8gx/n6sUVxm8H+ITyqWhCNeSzi+WS9cYk3FNpJcdTAwMDBO4+uDJ+Z5XHUwMDAx3Nr1q/HxvVx1MDAxOExcdTAwMWbA6Fx1MDAxYaN0M1x1MDAwZU5cdTAwMTfjcCFcdTAwMDRcdTAwMDWE7HRdXHLPR/v5rFvSZ1x1MDAxM9RuZC+uJmVr+KlcdTAwMTiHhfsnXHUwMDAxXHUwMDE0iGC5hVRHXHUwMDBmR0pcdFx1MDAwN1x1MDAwN9bVVq1cdTAwMWOSymU1ZZNJRPhu+Uhi5ltjSNrIsZyE+OeFqTjIP/5+JGzi8FBcdTAwMTOHSVx1MDAwMSGjW5g4XHUwMDE5Tt2+YWfnpYI9uK7kSnfUdNNNOMFltd1cdTAwMTKOXCJ4QfiO/ThFNaJX7Us4LpMmnrRLp6Uu1T9cdTAwMTXfcFx1MDAxMSbThEnGMFx1MDAxMfHpJno0Uko3XHUwMDE0p8++YYKo2YT45ptPat74ZoxcdTAwMTfo5PekzJlcdTAwMTcm2iCd/Fx1MDAxZWXOvMKAWdTagDuMQq1cdTAwMTdcdTAwMDJcdTAwMTFERMZ30Tx0Jvg0U3Snt4OcvK5nXHUwMDFmgPNcdTAwMTRl9zjyKeRcdTAwMTIsVrlcdTAwMDSRt3FJw2i2jMZcdTAwMDYljsSzXpiyXVx1MDAwNII8cTZJXHUwMDA0cSNXXHUwMDFmujnLMSynXHUwMDFkvMV0jJBcdTAwMTJbXHUwMDFmufv9Xs9yVTdO+5bjXHUwMDA2ayy+NztcdTAwMWP2p1x1MDAxZFNfe2j1zaFlXHUwMDAz7+tW6XP525elMCz+eP79f79urJ3xvUXvw3x3/OL/f1tcdTAwMTCS0MhcdTAwMWNMqbIuiE9cdTAwMWVeXHUwMDAy4dkwfyibR3JmN4lVMzK3+CEj0lxyQlx1MDAwMqBGiKSIXHUwMDEwTJDAS7vmXHUwMDExklCTXHUwMDEyS8aRVIVcdTAwMDDTQMfeXHShWGiUqqGmlFFcdTAwMGVcdTAwMTlbTozPgKVCXHUwMDAzgFx1MDAwMYw4UDWU6Vx1MDAxN8Qv8jRcdTAwMTVcdTAwMDbpbkPpfsJ3Y+3wN+x91t7tO2FcdTAwMWJcdTAwMTJcdTAwMTm86lx1MDAwZrtTso9lfNVWZI7ZoV65O3T7h5XLxlx0N8WMpVx1MDAxYt1qyDVIXHUwMDE0bqEgVGF8OVx1MDAxY1x1MDAwYlxyjEpNXHKAsp9cdTAwMDBkqlxuJ4GOvVx1MDAwZro3sC9ZZ18vXHUwMDA2Uk1B9Cf7plx1MDAxML4rtcl7Ua9cdTAwMWH1MHhcIqSklkFcdTAwMTBfXHUwMDAxrlx1MDAxNbL89Lhj2pmzXG4tzub8/EDKdKNcdTAwMTNcdTAwMDVjlFgyXG6wiIVANVdAXCJ2vH75XHUwMDEzgCHqr0hcdTAwMDaCODRMXHUwMDEwXHUwMDAxSL1cdTAwMTV4XHUwMDFlfz2zcsvhbZWP+eye1c/ynSq4M8YphyBJXHUwMDEzXHUwMDA0hVRcdTAwMTaoXHUwMDEw8idcdTAwMDemXHUwMDEwgm9EYJRPIWJcdTAwMWRIcE6gXHUwMDE08UF43yoj535UXHUwMDE43etNLLq5wqXMdtJccsJcdTAwMDBcdTAwMDbpXHUwMDFifVxuTdMghlx1MDAxZVx1MDAwZoNiXVx1MDAxMYVcdTAwMDQofZiTxFx1MDAxN16fXHUwMDA1aINT4bBaxSVxfkarPWNcXDuAp3VcdTAwMDeefyqnXHUwMDAyXHUwMDA0oVLNpbJEXHUwMDEwXHUwMDEw8aOmoocjrV5cdTAwMDVcdTAwMWFwKlD8JsGOdCr4TLulm1quXHQ3xlx1MDAxOHrG3qf3KsR3Ulx1MDAxYoP+MKlY3Fx1MDAxNybboGMh0JWgb0Hwzd5cdTAwMDX6amJcdI9cXERUQshcdTAwMTiLb2BcdTAwMTVg5bJ+Olx1MDAxOJ9cdTAwMDO7XGKn9cJ5XHUwMDBmXHUwMDFiYZ69lFx1MDAxMFx1MDAwYnxnZlx1MDAxMagpoflaZuFcXFxiKOGudro/vkVcdTAwMGJkbnv715eV7kGmXFxcdTAwMWI6vevs9f4nI5bQXHUwMDE1e45cdTAwMDVFSoWPTyzRw5FSYoGBXHUwMDA0XHUwMDBlifLKXHUwMDA20V7nXHUwMDE1iiFmRM0oP1x1MDAwZa0o/YYkxCovzLRBVlntSeKkXCLC92BcdTAwMDHiuZBcdTAwMTiMjz8+zvObXHUwMDA3t1Pu6sV5q3h2XHUwMDA3nUOeblJcdC5cdTAwMTns2FxcQVx1MDAxMCOKXHUwMDExTlx1MDAxY3pRrDKqzYwumffK1zfialrA34qoXftcXKyCI1hcdTAwMDUokWYwvqpcdTAwMTQ9XHUwMDFjKWVcdTAwMTXEPs5awVx1MDAxYmRcdTAwMWKtybayXHUwMDExIVx1MDAxMf79XHUwMDEwO2KVj9s2+Js1SIhUXphpg6Sy0pFcdTAwMDCnoPdmlPDkKIRjiimS8Vx1MDAxOeW0V7xouDhj9U7274Xsi059OEg3o+BUmSlqyJlgKPmg2vdE3WtFT4ZvtaCEXHUwMDBiNVxu8aN/zIvT+vW4Nep0ipel/FlLXGZlKZtuySNBS2K3ulxmXHUwMDA2XHUwMDFjMVx1MDAwMne79NpcdTAwMWJyflLPVb6NdFjusv3y9X6m9Z66zLvJdfiO1dClV09bXHUwMDE0gqMtlJno8UipMkN4XG5cdTAwMTdf1WxcIjGWaOdW8sfpM/9YyGoyXHUwMDFhzVx1MDAwYvNtUKNcdHQlcTtZhmfgkZhcdTAwMGJJJY6v1ZRn1ZPzQe58XHUwMDAys5lad1C7bN643XRzXHUwMDBifWdueZtWXHUwMDAzXHUwMDA15VxufeQ7UmvCYjpQqDotMCWSUVx1MDAxMt+bXGbztfk4XHUwMDBmOs3TSp7bl11sno9KaZ/8g5vU3ih50Zt54kU3KoVaMknhzoOTXzX1f+eRXHUwMDFkaKV65t2Cq8JcdTAwMTOHUFxiXHUwMDE4gnCL2Crn/Mhl41x0up8qXiPwuHFUb6VcdTAwMWaIXCJ1QCSYXCLOQPIpXG5+XHUwMDAy0ftsXHUwMDE55ZhcdTAwMThcdTAwMTLDdTGKXGKHXHUwMDFjkfj7XHUwMDAwukIviMKdPVx1MDAxYzxY01bdatl56aRcdTAwMWSKwaTvu4dcIuJYoUCAn0j8oZBcdTAwMTi+1lx1MDAwYphSTSXE8Zcmpjd6TlxcTMyLYeGKZmdcdTAwMTSc5WdcdTAwMTepR1wiSFx1MDAxYlx1MDAxMj3fPVx1MDAxN1xikc+5MPGdQ3FcdTAwMDecyFx1MDAxOZfKWGHxObHaXHUwMDE1XHUwMDE5nmv17Ubl5r7EblxusH5cdTAwMTm2/Tw1SITBZCZvXHUwMDBi/n9cdTAwMGYkMkyI9M5D+YnE9CExQIpcdTAwMWaBRCFcdMFcdTAwMDJusUu1kLlcdTAwMWZna1x1MDAwZi5t3F6WwZRfdEUp9ZxcdTAwMThcZtRMXHUwMDAxXHUwMDEyhVx1MDAwNFRcdTAwMTJcdTAwMDB/qqcpROJcdTAwMWKBXHUwMDE45jiLyPTqpVx1MDAxNlx1MDAxN1x1MDAwMG+xYnOCRqIxOlZk2pzamMt5heqjdC/ZY1x1MDAxMvSZIY1hSLmahryogLftXHUwMDBmb8FcdTAwMTZqoXVA+sZ06T1bS+8gXHSTUlmMO9+e+mbnXHUwMDE51J7HXHUwMDEzKbHyXHTdiyGn2uLfV19OkVx1MDAxNY+abbZW5XrVn+b2fYEwq860lWdYXHUwMDBmMF1vN1x1MDAxOFx1MDAxMsRcdTAwMTFGkHineng/N6d5XHUwMDA1XHUwMDFh91dir8Ooz49cdTAwMWJM2MKRXHUwMDA0Sn2KT5bZ8WS/djU+7d7P3KK4OyvhPK2mXHUwMDFio5RxTVx1MDAxMsmAxNzLrbv8lkfEQo0yylx1MDAwNGOcK+i+KTVmKGLZppyYvqaeMphJXHUwMDBlJVx1MDAxMezzh+9hjaghXHUwMDA38GkqJGKlVlx1MDAwNGK/Ppjmnfbvr1x1MDAxZovX9VbXnN2UYSQ5JZD7c/L7wYpcdTAwMTYpVr1iNY5cbljRfvDQY1x0cWi+TDVccnhcdTAwMTKyxXJPXHUwMDE3TzrtIyHvjknBNsrSunLcm3SjlWCuXHQlM4ChQIJBKjQpXHUwMDA0XHUwMDEy/lM0glx1MDAxOMVcdTAwMDTr5JVcdTAwMTk0xVx1MDAwNlbdoOZKgcFcIlx1MDAxOOHTgzQkJmU5XGZ/g7JsuvpcdTAwMWbOt5myXHUwMDEymu7rj4Z6LTpDm49cdTAwMTVli6Mz14aGhMnQXHUwMDFjn8wzciTfXCJvbUt2+XmenVx1MDAxNGYund/k6b5lneTSjUNFRJrStVxi84f5LFxik1x1MDAwYlxyMqWHeXRcdTAwMWFcbkTaVKag8Tog0uVcZlx1MDAxOLnyXHUwMDAzXHUwMDE1ayvF6HOuwb5cboiXylwiZORcdTAwMGanptvjsCCxhCgypOkkIYjCl3k4UlxmS7fJRdYzjFF7eNq+v0ZH7VnnMH9+Xkz56ThcYiFcclxiL8lcdTAwMTdPJ1x1MDAxNWJEXHUwMDE58vP0d1x1MDAwZsCqXHUwMDEy2Vx1MDAxZDJhaPOJwlx1MDAxMIZH5VxiRlx1MDAxMJBbXHUwMDFjjOhMK7Vb48TNXHUwMDE0ji7MXHUwMDFjL/dy345TnnVcdTAwMTdpmFx1MDAwN9guRSBUyohcdTAwMDCUf79bvtZBmLdM29ghXG7D209cdTAwMTKGmIZvI0ZcdTAwMTIpg3ObpEenJXxQmF/0Olx1MDAxM3xWXHUwMDFjtLm5X7ImKcchI1x1MDAxYcZcdTAwMWLoXHUwMDEwSqhcdTAwMDGqjLIoJFx1MDAxMklcdTAwMTVMXonE5chHIFFZ7oRKvvts2Fx1MDAxZofE/b7TXHUwMDFkO01XyeJcdTAwMWbOP7MnXHUwMDA3//poLEb1IFk0hifKlZhRiuRcdTAwMTZcdTAwMDerdkFfiLtqQc+A4oO+3ymRxmnYkUMpQSPEWFx1MDAxM2pcdTAwMTSCYFR8qFx1MDAxMYAwibJcdTAwMGXfXHUwMDA0xqVRXHUwMDEwXHUwMDA1Rlx1MDAwZVXfKP2BjMNcdTAwMDNr5INC5fzDsVx1MDAxONGBRKFcdTAwMThxplx1MDAxZMVcdTAwMDB4LyH+viHs5Mgk705yWVev3LZGg8OqbadcdTAwMWKKStQ0L/X+2lJN4lhkyzWyyMhcdTAwMDBcdTAwMGUhglxm/EBgPDHb+lx1MDAxM1x1MDAxME4q1Vx1MDAwZodiaPNJXHUwMDAykYDwo75cdTAwMDBcXHi24u9cci+XukajMIOy+1xyzibXtcn0sJxJOVx1MDAwZT24IVx1MDAxOEQhXHUwMDE2xPM/rlx1MDAxNlx1MDAwNFFoMs5fe/RcdTAwMTePtV5KvZVcdTAwMWGFwlx1MDAxZlxihN/ux7qt3t5cdTAwMWZOZWBcdTAwMGX1XHUwMDFkmIpRPUhcdTAwMTSKvsjktXNeMVx1MDAxMELSLfbSXlhHR/Vh3zmf3bfqSFx1MDAxN52T42I53VjEdElNXHUwMDFmXHUwMDAywnhqqUJcdTAwMDBcdTAwMTD0O97Qvsl52GsoXHTsWIPdwTC6XHUwMDBmSVx1MDAwMpHi8FxiOaxAXGI5RfGBiGS90u+fVW/71dnAuujq9dN6M91AREyD3sGzKWVFpOAoXHUwMDAwIclng0tcdTAwMGZcIs/VmzJ3XHUwMDA3xtDmXHUwMDEzdWKIiJBxgqhcdTAwMTKhLY5O6dCmcTO9OG2wzkNcdNVGLFx1MDAwN89AunGo7EBccmBcbmkwriY1/nxcdTAwMDS9I43wXHUwMDBmtHx6MW44prtcdTAwMTN3fkjTieqkNEInXHUwMDA1jFx1MDAwYlx1MDAwMbbJXHUwMDE4WexcdTAwMTa6/L40bFx1MDAxNNqGcdNcdTAwMWWXKE75UdBcZlx1MDAxMY3uXHUwMDA2gixcdTAwMWVcdTAwMDSp581lXHUwMDFmkE4sPVx1MDAxODzVXUVzzk5AXHUwMDE41vZWKPzlac/Hnj5cdTAwMThcXLhqLFXZIyb3JpY5zW2SksXHy8O2wLBcdTAwMDdcdTAwMTbTey1//vXLX/9cdTAwMDdzm3XvIn0=and or not <= in == dport 1024 :ip #type 10.0.0.0/8 /zeek.*/ Meta Extractor Uint64 Value Type Extractor Field Extractor Conjunction (AND) Disjunction (OR) Negation (NOT) Equality Operator Membership Operator Range Operator Subnet Value Pattern Value
When written out, it looks like this:
(dport <= 1024 or :ip in 10.0.0.0 / 8 ) and not (#schema == / zeek. */ )
In this example, the predicate operands dport
, :ip
, and #schema
represent
extractors that resolve to a set of matching fields at runtime.
Let's take a look at the expression components in more depth.
Connectives There exist three logical connectives that connect sub-expressions:
and
: the logical AND between two expressionsor
: the logical OR between two expressionsnot
: the logical NOT of one expressionPredicates A predicate has the form LHS
op
RHS
, where LHS
denotes the left-hand
side operand and RHS
the right-hand side operand. The relational operator
op
is typed , i.e., only a subset of the cross product
of operand types is valid.
An operand is either an extractor or a value . The
typing can often only inferred at runtime when schema information is available.
Relational Operators The following operators separate two operands:
<
: less than<=
: less equal>=
: greater equal==
: equal to!=
: not equal toin
: in (left to right) !in
: not in (left to right) ni
: in (right to left) !ni
: not in (right to left) Type Compatibility The table below illustrates a partial function over the cross product of
available types. Each letter in a cell denotes a set of operators:
E : equality operators ==
, !=
R : range operators <
, <=
, >=
, >
M : membership operators in
, !in
, ni
, !ni
Bool Int64 UInt64 Double Duration Time String Pattern IP Subnet Enum List Bool E M Int64 ER ER ER M UInt64 ER ER ER M Double ER ER ER M Duration ER M Time ER M String EM EM M Pattern EM EM M IP E EM M Subnet EM EM M Enum E M List M M M M M M M M M M M EM
An extractor retrieves a certain aspect of an event. When looking up an
expression, Tenzir binds the extractor to a specific record field, i.e., maps
it to the corresponding numeric column offset in the schema. Binding an
expression implicitly creates a disjunction of all matching fields. We find
that this existential qualification is the natural user experience when
"extracting" data declaratively.
Tenzir has the following extractor types:
Field : extracts all fields whose name match a given
record field name.
Type : extracts all event types that have a field of a
given type.
Meta : extracts metadata describing the event instead of
the actual values contained in it
The diagram below illustrate how extractors relate to each other:
eyJ2ZXJzaW9uIjoiMSIsImVuY29kaW5nIjoiYnN0cmluZyIsImNvbXByZXNzZWQiOnRydWUsImVuY29kZWQiOiJ4nNVaW1ejSFx1MDAxMH73V+RkXkem71x1MDAxN9+MOjo7O6NuvKzuzPEwXHUwMDAxXHIrXHUwMDAxXHUwMDA0jNE5/vctUGkgXHUwMDA2Y4xuzENcdTAwMDJ9re6ur+qr6vxearXa6XXktldabXfUs33Pie2r9sesfOjGiVx1MDAxN1x1MDAwNlBF8vckvIx7ect+mkbJyqdPdlx1MDAxNFmml9VcdTAwMGJcdTAwMDd3PV3fXHUwMDFkuEGaQNt/4L3V+p1/l+ZK3VGat81LzUxC11x1MDAwYr+HQT4pVlx1MDAwNGHGXHUwMDExXHUwMDE2RVx1MDAwYi9Zh5lS14HqU9tPXFxTk1x1MDAxNbVcdTAwMGa2d3dFnEZHnTA9Xt87idG3v1x1MDAwZc2sp57vd9NrP1x1MDAxNyhcdGFcdTAwMTGmLknj8Nw99Jy0n81dK5/UK1x1MDAwZS/P+oGbJJU+YWT3vPQ6K0OoKLWDs3xcZlMygjdKiYUll1xuXHUwMDEzXlRkXZnUXHUwMDE24VQjUaq4k2Yt9MM4k+ZcdTAwMDPKP0aeX3bv/FxmhFxunKJNXHUwMDFh20FcdTAwMTLZMVx1MDAxY49pd/WwTkKLsr7rnfXTTCRzJImb77VcdTAwMTJKU6IkKSqySaIvTn7gP8vbXHUwMDExOPfbXHUwMDExXFz6vpErq9ioK0lZUUqn+Ef36+bm0Vcx3LpEXGZ3/NVBd51cdTAwMTSyV7TKjuPwql3U3N4/XHUwMDE5iS4jx77TXHUwMDE3LCTmWGsulFRFve9cdTAwMDXndWH9sHf+iIqdhkHa9W5y1Vx1MDAxNJXSz/bA87MzM5uZqztcYthqLbf6YZJcdTAwMDb2wF35XHUwMDExZK93qFopXHUwMDFmWuKCXHUwMDFj2cCUV0ZY9b2zXGYkbd89raIn9Vx1MDAwMIRFdVx1MDAxYUamtlx1MDAwN1x1MDAxMtkwXFz8xamvLIy9My+w/b2npcu73H58XHUwMDFlmJWcXHUwMDA0Zik110xcbqP8T2H5YsfRXHRcci754Obzcq+Tflx1MDAxZI22vcXGMmNcdTAwMDYgOYRcdTAwMTWqzW2Qi5Vg9q/ZkSuNXG42XHUwMDAwXHUwMDE3LIsgXHUwMDE0Yf42yL22l9F3frG9umVcdTAwMWatb1xmXHUwMDBmd7ona5vzQS5cdTAwMDE7h7kyXHUwMDA38/rIhSPzgrNcdTAwMWaB7TjxYqC1XCLRbFxinVx1MDAwNFBBNOWIXGI2NUCHa8fp8CveXHUwMDFmXHUwMDFlb/F/l8OLzn68t73YXHUwMDAwpcy851x1MDAwMOWyNvf8XHUwMDAwKsxWXHUwMDE3XHUwMDAwJSV7cFx1MDAwN1AtiVJCI7PtL8Vnk++jipG3832x21x1MDAwYmPncUdXXHUwMDFh5S2g8yBKI2buTNIjoMFCTIJcckZcXFx1MDAxMs2Vmt6xNdObeeDGXHTTbPp5XHUwMDAyXHUwMDA3MUtiJatcdTAwMTSVa2xRJapcdTAwMTXzpqiaWUyUJyjQxKWlSIU3P3g9XHUwMDA1bINgRJ/j9Yw7e1BcYnJfcjtcdTAwMDPYwF1hXG6MZ1x1MDAxNrAlqVx1MDAxZKdcdTAwMWQvcMDWV1x1MDAwNbtcdTAwMGazcn1vXHUwMDFm6Vx1MDAwMHX6XVx1MDAxNlx1MDAxZPWP3Vx1MDAwMb05QM7JqORsXHUwMDAxnr3LTEpkYcIoXHUwMDEyWmupXGLVIGCp1ZlcdTAwMWRla7VcdTAwMDTOz3FszW7gPC1Lc+BVkmVcdTAwMTlZlFNcdTAwMGVhneSEgeVcdTAwMTN8TFx1MDAxNswt0DfxmDC+naRr4WDgZVxuvlx1MDAxM3pBWt/TfPNWMyT3XXvMLsBiynWged6dwlx1MDAxNy2ibNAq0TFPLVx1MDAwM4v8pXj++fHR1lx1MDAxMllIU8nLnZZheVx1MDAxYbMsrHuqe1318+6F1pveS+Xf5zJcdTAwMDLaQFx0WDZTyZE9ZdqaVXIhKVx1MDAwMSHYwsAsRS385lJavGp45m3ZSmS2sGlsjCFgQVx1MDAxMKWckPlRhCZcbv9+g2+zm1x1MDAwZlx1MDAwNOSz5/rOj2BjXHUwMDA0x9BLw1x0JJ5V+lx1MDAxNVSjXHUwMDA3u+XGr8BFxqSaXHS2nNZcdTAwMGJcdTAwMGJKXHUwMDAy8Vx1MDAxZSaKYuP9nsLt1rGzjfdcdTAwMGVcdTAwMGVcdTAwMGX3d6+7W+ejYTLaQIuNW0pqVL50jnWg9jSxif2CWNtEXHRcclSe5Fx1MDAxYi/YXHUwMDFiZck6nVx1MDAwM7R2cXHRu+7qw/PVwapwXHUwMDBmbuZcdTAwMTZrXHUwMDEz4C7G8r12pDC4doew5ueFXG6vXHUwMDA20EKamYCJ6cQkmFKSKkrR9DH234kz+HW9XHUwMDFmuZc01VvL36LhXHUwMDA1Xl9wYCpcbvxGalGPXHUwMDE2KFx1MDAxMVx1MDAxNuhcdTAwMTZltMGnvlx1MDAxNKrT+VSBITigTLyNS31dpGJVXsdru9RvbmovnEetXHUwMDBi1YjbyVG+nuxSXHUwMDE5l1pcdTAwMDJrn1x1MDAxZbnNh76gUb6CeF5IJVx1MDAxOGZcdTAwMDTjXHUwMDEyxcjxy7CFuORcdTAwMTRRwagu5brnXHJilqVcdTAwMTVcYiFcdTAwMWMpxVx1MDAxMVKP3FGBpCBcdTAwMDGD+JFyXHUwMDAxUo1nvommQlx1MDAxM1JKzPxcdTAwMGY5gMyNilnAOWVcdTAwMGWg2T+0yjlcdTAwMDAhMFx1MDAxNUBcbokkXHUwMDE4NkaVXHUwMDFhPaRcdTAwMDA4R1x1MDAxOFx1MDAwMm8kXHUwMDE12Fx1MDAxNI3Hlj5VKqCZS7YqqVx1MDAwMKZcdTAwMTFcdTAwMWOzVDzLXHUwMDA1XHUwMDEwPS5cdTAwMTNcdTAwMWVcdTAwMTfinaVcdTAwMDCWWaU9maLDJO3PPmN6b4ZbKv8++16g4Vx1MDAxYV4jXHUwMDAxtuBcdTAwMTms5U/nRu5eXHUwMDFjbP7hXGbCtZ3zUTD6fpIsNmtcdTAwMDGlt1x1MDAxNNWqQk7yPFx1MDAwMCGW5lx1MDAxNDdcXMK/9KZgOs5cdTAwMDKHzlx1MDAxMZGle7/3e5OHXHUwMDEx0WaBr81Z9kDQheMsdaFm4yxcdTAwMTRPvJlcdTAwMDCzKjmiZPrsXfOZLyZlXHUwMDExVFuMMUGEoFKVQq9cdTAwMWO7WFhSNf+D5qXghUjCgmBcdTAwMDesJM5cdTAwMDVAhlx1MDAxNpmLXG40lkq8JymYM4VcdTAwMThlz1x0ROZ/UTErXHUwMDFhpyQpze6gQlLAy3CumNZcdTAwMTL4XHUwMDA3Nf9nKFxiXHUwMDAxhJCinDt/Jjlp/lNJlZygzO9RXHQ0kmFwXHLykYtcbmBVLHdcdTAwMTnvnqVwalx1MDAwMfEjgJiaXHUwMDE239U+KPCT40yGQ15NmYVqXHUwMDAzVWnL0v3QbTuKuikoanGY7aHnXnXG0fnhNP9kXHUwMDA2NN/+zE65OU5ul27/XHUwMDAzl6ZcdTAwMTJcdTAwMGUifQ== - hostname: - source: string addr record: Field Extractor myevent: Meta Extractor Type Extractor
Field extractors have the form x
or x.y.z
where x
, y
, and z
match on
record field names. The access fields in nested records. Using a type name as
leftmost element before a .
is also possible.
A field extractor has suffix semantics. It is possible to just write z
to
access x.y.z
. In fact, writing z
is equivalent to *.z
and creates a
disjunction of all fields ending in z
.
Examples ts > 1 day ago
: events with a record field ts
from the last 24h hourszeek.conn.id.orig_h in 192.168.0.0/24
: connections with source IP in
192.168.0.0/24orig_bytes >= 10Ki
: events with a field orig_bytes
greater or equal to
10 * 2^10.Type extractors have the form :T
where T
is the type of a field. Type
extractors work for all basic types and
user-defined aliases.
A search for type :T
includes all aliased types. For example, given the alias
port
that maps to uint64
, then the :uint64
type extractor will also
consider instances of type port
. However, a :port
query does not include
:uint64
types because an alias is a strict refinement of an existing type.
Examples :timestamp > 1 hour ago
: events with a timestamp
alias in the last hour:ip == 6.6.6.6
: events with any field of type ip
equal to 6.6.6.6:uint64 > 42M
: events where uint64
values is greater than 42M"evil" in :string
: events where any string
field contains the substring
evil
Meta extractors have the form #extractor
. They work on the event metadata
(e.g., their schema) instead of the value domain.
#schema
: the human-readable name of the schema#schema_id
: the unique fingerprint for the schema#import_time
: the ingestion time when event arrived at the serverExamples #schema == "zeek.conn"
: events of type zeek.conn
"suricata" in #schema
: events that have suricata
in their schema name#import_time > 1 hour ago
: events that have been imported within the last
hourThere are three short forms for defining predicates succinctly. They are merely
syntactic sugar and can be used whenever a predicate is expected. The following
table shows how values, field extractors and type extractors are expanded.
Short Form Expansion Example value
:type == value
"needle"
→ :string == "needle"
field
field != null
header.req
→ header.req != null
:type
:type != null
:ip
→ :ip != null
The first form requires the event to contain a field with the given
value. This allows for quick type-based point queries, such as
(6.6.6.6 || 10.0.0.0/8) && "evil"
. The second and third short forms
test for the existance of a field or type. They are useful to filter
out events with missing information.
Values of type subnet
expand more broadly. For example, the subnet
10.0.0.0/8
expands to:
:subnet == 10.0.0.0 / 8 or :ip in 10.0.0.0 / 8
This makes it easier to search for IP addresses belonging to a specific subnet.
Values Every type has a corresponding value syntax in the
expression language.
Literal Example none
null
bool
true
, false
int64
-42
, +3
uint64
0
, 42
double
-0.7
, 1.337
duration
-3us
, 24h
time
now
, 1h ago
, 2020-01-01+10:42:00
string
"foo"
pattern
/fo*.bar$/
, /^\w{3}$/i
ip
::1
, 10.0.0.1
, 2001:db8::
subnet
::1/128
, 10.0.0.0/8
, 2001:db8::/32
list
[1, 2, 3]
, []
record
<x: a, y: b, z: c>
, <a, b, c>
, <>