Sigma

Experimental Feature

This is an experimental feature: the API is subject to change and robustness is not yet comparable to production-grade features.

VAST has native support to execute Sigma rules. In short, this means you can provide a Sigma rules instead of writing VAST query expression. For example:

vast export json < sigma-rule.yaml

Sigma defines a YAML-based rule language along with a compiler that transforms rules into the native query languages of SIEM systems. The project ships with collection of detection rules that apply to endpoint and network log telemetry.

Usage

VAST automatically detects whether a query is in "Sigma syntax" or a VAST expression. Therefore, running a query with a Sigma rule is no different from running an ordinary query.

Using the vast export command, the following options for querying are available:

vast export <format> <expression>

and

vast export <format> < sigma-rule.yaml

where <format> represents an output format, such as json or csv, <expression> the actual query string, and sigma-rule.yaml a file containing an <expression>. In principle, you could join a Sigma YAML file into one big string and pass it as <expression>, but more practically is file-based execution.

Search Identifiers

Sigma rules require a detection attribute that includes a map of named sub-expression called search identifiers. In addition, detection must include a final condition that combines search identifiers using boolean algebra (AND, OR, and NOT) or syntactic sugar to reference groups of search expressions, e.g., using the 1/all of * or plain wildcard syntax.

Consider the following Sigma detection embedded in a rule:

detection:
foo:
a: 42
b: "evil"
bar:
c: 1.2.3.4
condition: foo or not bar

VAST translates this rule piece by building a symbol table of all keys (foo and bar). Each sub-expression is a valid VAST expression itself:

  1. foo: a == 42 && b == "evil"
  2. bar: c == 1.2.3.4

Finally, VAST combines the expression according to the condition:

(a == 42 && b == "evil") || ! (c == 1.2.3.4)
Rich YAML Typing

Because VAST has a beefed up YAML parser that performs type inference, the YAML snippet c: 1.2.3.4 is parsed as a key-value pair with types string and address. This means that we get the rich type system of VAST for free.

Taxonomy

Similar to VAST's taxonomy feature, Sigma also comes with a taxonomy to facilitate rule sharing by standardizing field names of the supported data sources.

Missing Definitions

VAST currently does not ship with a taxonomy to transparently map the canonical Sigma fields to an equivalent in VAST. We will ship the missing mappings in the future.

Comparison

VAST and Sigma have many commonalities. They both support flexible construction of search expressions using boolean algebra (AND, OR, NOT) and offer multiple ways to define predicates and sub-expression. But there also exist differences in expressiveness and intent. This section compares the two systems.

Expressiveness

The majority of rule definitions include combinations of exact string lookups, substring searches, or pattern matches. Sigma uses modifiers to select a concrete operator for given search predicate. Without a modifier specification, Sigma uses equality comparison (==) of field and value. For example, the contains modifier changes the operator to substring search, and the re modifier switches to a regular expression match.

Sigma currently lacks support for ordering relationships, such as less-than comparison of numerical values, e.g., x < 42 or timestamp >= 2021-02, whereas VAST offers relational operators (<, <=, >=, >) for numeric types.

Compatibility

VAST's support for Sigma is still in the early stages and does not support the full language specification. The following features are currently unsupported:

  • Certain value invariants:
    • VAST does not offer case-insensitive search, whereas it's the default in Sigma
    • Interpretation of string values containing * and ? wildcards
  • The following modifiers:
    • re
    • base64
    • base64offset
    • utf16le / wide
    • utf16be
    • utf16
  • TimeFrame specification
  • Aggregation expressions
  • Near aggregation expressions

Focus on Endpoint

Sigma predominantly offers rules with a focus on endpoint data, such as Sysmon telemetry. While there exist rules for network-based detections (e.g., for DNS queries, SMB events, and Kerberos traffic), they receive less attention.

As of Februrary 2021, the rules directory includes a total of 709 total *.yml files compared to 36 files in the network directory:

find rules -name '*.yml' | wc -l
709
find rules/network -name '*.yml' | wc -l
36

That is, network-based rules account only for 5% of the total rules. This illustrates the emphasis of the community and project authors, who have strong background in endpoint detection.

VAST's history emphasizes network telemetry, with native support for PCAP, NetFlow, and full support for network monitors like Zeek and Suricata. By natively supporting Sigma in VAST, we are looking forward to offer a platform with detection capabilities on both ends of the spectrum.