Expressions

VAST's query language is designed for effective subsetting of data. The syntax borrows from the terseness of awk and from network-centric focus of tcpdump, but enhanced with a rich type system.

Expressions

A query is a boolean expression that declaratively defines the subset of interest. Sub-expressions can be connected as conjunctions (&&), disjunctions (||), and negations (!). Expression operands are either sub-expressions or predicates at the leaves.

The following example shows an abstract syntax tree (AST) along with the corresponding expression.

Let's take a look at the individual components in more depth.

Predicates

A predicate has the form LHS op RHS, where LHS denotes the left-hand side operand and RHS the right-hand side operand. The relational operator op is typed, i.e., only a subset of the cross product of operand types is a valid syntax. An operand is either an extractor or a data value. Operands are always typed, but for extractors the type can sometimes only be inferred at query runtime. In this case, the type check also takes place lazily.

Operators

The following operators separate two operands:

SymbolName
<less than
<=less equal
>=greater equal
==equal to
!=not equal to
inin (left to right)
!innot in (left to right)
niin (right to left)
!ninot in (right to left)
~match
!~not match

Type Compatibility

The table below illustrates a partial function over the cross product of available types. Green cells represent a valid combination of LHS op RHS for the given set of operator classes.

Type Compatibility

Extractors

An extractor retrieves a certain aspect of an event. VAST has the following extractor types:

  1. Field: extracts all fields whose name match a given record field name.

  2. Type: extracts all event types that have a field of a given type.

  3. Meta: matches on the type name or field name of a layout instead of the values contained in actual events.

Field Extractor

Field extractors have the form x or x.y.z where x, y, and z match on record field names. The access fields in nested records. Using a type name as leftmost element before a . is also possible.

A field extractor has suffix semantics. It is possible to just write z to access x.y.z. In fact, writing z is equivalent to *.z and creates a disjunction of all fields ending in z.

Examples
  • ts > 1 day ago
  • zeek.conn.id.orig_h in 192.168.0.0/24
  • orig_bytes >= 10Ki

Type Extractor

Type extractors have the form :T where T is the type of a field. Type extractors work for all basic types and user-defined aliases.

A search for type :T includes all aliased types. For example, given the alias type port = count exists, then the :count type extractor will also consider instances of type port. However, a :port query does not inclucde :count types because an alias is a strict refinement of an existing type.

Examples
  • :timestamp > 1 hour ago
  • :addr == 6.6.6.6
  • :count > 42M
  • "evil" in :string

Meta Extractor

Meta extractors have the forms #type or #field. They work on the layouts of events instead of the value domain.

The #type form matches on the event layout name, or event type, hence the name. Anologously, the #field form matches on the field names of events.

These forms are useful when you're interested in exporting all events of a certain type or those containing a particular field independent of the event's content.

Examples
  • #type == "zeek.conn"
  • "suricata" in #type
  • #field == "community_id"

Value Predicates

Predicates with type extractors and equality operators can be written tersely as value predicates. That is, if a predicate has the form :T == X where X is a value and T the type of X, it suffices to write X. The predicate parser deduces the type of X automatically in this case.

For example, 6.6.6.6 is a valid predicate and expands to :addr == 6.6.6.6. This allows for quick type-based point queries, such as (6.6.6.6 || 80/tcp) && "evil".

tip

Value predicates of type subnet expand more broadly. Given a subnet 10.0.0.0/8, the parser expands this to:

:subnet == 10.0.0.0/8 || :addr in 10.0.0.0/8`

This makes it easier to search for IP addresses belonging to a specific subnet.