Expressions

VAST's query language is designed for effective subsetting of data. The syntax borrows from the terseness of awk and from network-centric focus of tcpdump, but enhanced with a rich type system.

Expressions

A query is a boolean expression that declaratively defines the subset of interest. Sub-expressions can be connected as conjunctions (&&), disjunctions (||), and negations (!). Expression operands are either sub-expressions or predicates at the leaves.

The following example shows an abstract syntax tree (AST) along with the corresponding expression.

Let's take a look at the individual components in more depth.

Predicates

A predicate has the form LHS op RHS, where LHS denotes the left-hand side operand and RHS the right-hand side operand. The relational operator op is typed, i.e., only a subset of the cross product of operand types is a valid syntax. An operand is either an extractor or a data value. Operands are always typed, but for extractors the type can sometimes only be inferred at query runtime. In this case, the type check also takes place lazily.

Operators

The following operators separate two operands:

SymbolName
<less than
<=less equal
>=greater equal
==equal to
!=not equal to
inin (left to right)
!innot in (left to right)
niin (right to left)
!ninot in (right to left)
~match
!~not match

Type Compatibility

The table below illustrates a partial function over the cross product of available types. Green cells represent a valid combination of LHS op RHS for the given set of operator classes.

Type Compatibility

Extractors

An extractor retrieves a certain aspect of an event. VAST has the following extractor types:

  1. Attribute: matches fields that have a particular attribute value.

  2. Type: extracts all event types that have a field of a given type.

  3. Field: extracts all fields whose name match a given record field name.

Attribute Extractor

Attribute extractors have the form #x where x is the name of an attribute.

For example, #timestamp extracts the event timestamp and requires an operand of type time. A predicate with a #timestamp extractor only considers event types that have a field with the #timestamp attribute.

Examples
  • #type == "zeek.conn"
  • #timestamp > 2 days ago

Type Extractor

Type extractors have the form :T where T is the type of a field. Type extractors work for all basic types.

Examples
  • :addr == 6.6.6.6
  • :count > 42M
  • "evil" in :string
tip

Predicates with type extractors and equality operators can be written tersely as data predicates. That is, if a predicate has the form :T == X where X is a value and T the type of X, it suffices to just write X as query. The predicate parser deduces the type of X automatically in this case.

For example, 6.6.6.6 is a valid predicate and expands to :addr == 6.6.6.6. This allows for quick type-based point queries, such as (6.6.6.6 || 80/tcp) && "evil".

Field Extractor

Field extractors have the form x or x.y.z where x, y, and z match on record field names. The access fields in nested records. Using a type name as leftmost element before a . is also possible.

A field extractor has suffix semantics. It is possible to just write z to access x.y.z. In fact, writing z is equivalent to *.z and creates a disjunction of all fields ending in z.

Examples
  • ts > 1 day ago
  • zeek.conn.id.orig_h in 192.168.0.0/24
  • orig_bytes >= 10Ki