The Community ID is a
standardized flow hash that generates a unique identifier for a network
connection. Since it's implemented across multiple tools, it allows for tracking
events associated with the same connection across different systems. This is
particularly valuable for correlating network and endpoint events.
Elastic just released their new pipeline query language called
ES|QL. This is a conscious attempt to consolidate the language zoo in the
Elastic ecosystem
(queryDSL,
EQL,
KQL,
SQL,
Painless,
Canvas/Timelion).
Elastic said that they worked on this effort for over a year. The
documentation is still sparse, but we still tried to read between
the lines to understand what this new pipeline language has to offer.
Our Tenzir Query Language (TQL) is a pipeline language that works
by chaining operators into data flows. When we designed TQL, we specifically
studied Splunk's Search Processing Language (SPL), as it generally leaves
a positive impression for security analysts that are not data engineers. Our
goal was to take all the good things of SPL, but provide a more powerful
language without compromising simplicity. In this blog post, we explain how the
two languages differ using concrete threat hunting examples.
