yara Executes YARA rules on byte streams.
Synopsis yara [-B|--blockwise] [-C|--compiled-rules] [-f|--fast-scan] <rule> [<rule>..]
Description The yara
operator applies YARA rules to
an input of bytes, emitting rule context upon a match.
eyJ2ZXJzaW9uIjoiMSIsImVuY29kaW5nIjoiYnN0cmluZyIsImNvbXByZXNzZWQiOnRydWUsImVuY29kZWQiOiJ4nO1aW1PbyFx1MDAxMn7Pr6A45zFo537ZqvNcdTAwMTCTYMgm7Fx1MDAxMu6styjZlm1cdTAwMTFZdixxcVL89/ONXHUwMDFjrFx1MDAwYjJ2XHUwMDAy2aW24kCQ59LdM939dfdovrxYW1tPp+Ng/de19eCm40dhd+Jfr7907VfBJFx0RzG6WPY9XHUwMDE5XU462chBmo6TX3/5xVx1MDAxZo+9fJbXXHUwMDE5XHJnM4MoXHUwMDE4XHUwMDA2cZpg7J/4vrb2Jfu/wCtcbuMgXHUwMDFim7XmnKhiutq8O4oztpRcdTAwMTBjOCFMzEeEyWvwSoMuunt+lFx1MDAwNHmPa1o38Zu37+3pqfx8M+nZT93zV/7Rh5xvL4yi/XRcdTAwMWHNXHUwMDE25XdcdTAwMDaXk4JUSTpcdTAwMTl9XGaOw246cNwr7fN5yVxiy89nTUaX/UFcdTAwMWMkSWnOaOx3wnQ6W8W81Y/7XHUwMDE5jbzlxo3g+Vx1MDAwMrNcdTAwMTlaVnhvjqLRxPH+XHUwMDBmXHLcv5x72+987EOEuJuPXHRs0Fx0evmY668r4lx1MDAwNUlcdTAwMDZB2Fx1MDAxZqSOXHUwMDE5NZ5mXFxcdTAwMTKiJLGU88K6g2yfXHUwMDE5MZxLwfi8wzFcdTAwMWPvdDN1/5Vv7sRcdTAwMWZcdTAwMDY7bkZ8XHUwMDE5RcX9ibtf96fU0XZcdTAwMWRvXG6Wk5O6XHUwMDFjd/2ZkqmyRlx1MDAwYkNcdTAwMDUnis37YU1cdTAwMWar5KJR52ONXSSpP0lcdTAwMWJh3Fxy4351Slx1MDAxMHdcdTAwMTf0RH6Sbo6GwzCFXHUwMDE4f4zCOK2OyOi+mkxG14PAv7doUF7YN3bkcldxn/xpLbeN7Mv8+a+XtaNcdTAwMTXxSPFDXz7UuYxaifd9y1g2nXHrMSVccqGUXGIrrH5cdTAwMWO5gr3OVvPN/HlJXHUwMDAwTzBKjFx1MDAxNJpKRlx1MDAxOCFcItig4pu2pCDBi+Lf25f1qDdcdDrpzOdroE8oWW29Qz5rXGKRnJPVge/itz3L+qElp39cdTAwMDRcdTAwMTfJ9uX0XCI62VtcdTAwMDB8XHUwMDE1XHUwMDAwezzs5bDyLbDHiPJcZqdWKaZcdTAwMTgwMMeXadarPcOU5lpcdTAwMTDBhOaLMTEgnHK6XHUwMDA0XHUwMDEzK2PmmEg9ZtDDjVFSa0VyKeZcdTAwMTAptedsxlxiqjVhWlx1MDAxNCSZISRcdTAwMDU2KVhUQaFFiJxcdTAwMWLN+uut3oRdXHUwMDFj/D65eHvuN96zt6a3XHUwMDFmrt+Z0vNE0lx1MDAwN1xyfFFYZ1SqhWFcdTAwMWQgoKWUlq1s3Vx1MDAxYifJ4eVcdTAwMWbvXHUwMDBlTrZcdTAwMGZZ78B/J496m78/c+vmwrNcdTAwMDKrlJQrS1x1MDAwYqZxZ916XHUwMDA2VVxuYGXpQutOJ36cjP1cdFT7sIX3ss99XHUwMDBiZ9Kj2nKwUZCDKFFr4FxulsGZ5Fx1MDAxYX+EXHUwMDE2VVx1MDAwYpdMUy4tqc9cdTAwMDF+hIF/yYl+NTb2teX2Z1x1MDAwNvGUXHUwMDE5XHUwMDA08Vx1MDAxOOGaWCU0UUQjaJaipvaY1k79gjKBrVTL6G0oT1xuorVmXHUwMDEyUZhrZcrkqOOCXGJMXHUwMDE1d2pZSk57VmlpgVx1MDAxOUQpKbgqkqPEs4BkuKMyXHUwMDEwXHUwMDFjjrSUnvBQVFhutDFcZlx1MDAwMYaWklx1MDAxNMY9alx1MDAxMXCIsejjVq5AzlIrXHL2TUlqmCyJx5THqLVaW1x1MDAwNDDKjFlKjoJcdTAwMWUxXHUwMDE0W1x1MDAwNyFcdTAwMTSCSomc8aRVcENhXHUwMDE1XHUwMDA3gi7PXjakIyeYkVJYIC6xRXqcea5TWeiWXHUwMDAwkpeLp5yxXHUwMDE4jFx1MDAxNVxi3Jqr0mo5mCm3ddxcdTAwMWHsrzLLd49yT1xiilx1MDAxZikkgzp0yVhcdTAwMDRMiVKEXHUwMDEyWIpVRKjl61x1MDAwNUFcdTAwMDepTlx1MDAxYjBWXHUwMDA09ZIxS+JBXHUwMDE12F1NwJNjXHUwMDAxS1x0XHUwMDFhZJRcdTAwMDaRXHUwMDFkyShxiSWr0ONcZlx1MDAxZlx1MDAwMW1wY+XyXHUwMDA1W49AbXAjaFx1MDAxOHRtaVx1MDAwM4HD2Fx1MDAwN2jLMmyjsnYpOYZcZl/A+iAhsVx1MDAxYbmtrNCr4PpygrA/LSGdpNZATiFKXHUwMDA0mYeCXVx1MDAxMIosXlx1MDAxMki6nFx1MDAxZfcso1hcdTAwMTSWyinBY0nBwnNJXHUwMDE3Mlx1MDAwMlx1MDAwMt1LsoqCtVx1MDAwN5DisFx0+JqUypRcYnKwIyDJqWZcdTAwMDa4JZZrhDGYoEMsRGtOpWQli0F+aLlcdTAwMDBUcI5tXHUwMDExnK+wZOqBinBGg4DJyoCF/TXQXHUwMDAzvM6da8CR9CqQoFx1MDAwMFlcYmCCM22lVlx1MDAxNcSSRDMhXHUwMDFjQ2BccltcdTAwMDFjjIedhjFrrIcyVsJcdTAwMDRcdTAwMGKNuPLIVUhcdTAwMTLbvHy9XHUwMDE0oKBcdTAwMTE8OMvcTvKSRpiHXHUwMDE0XHUwMDFlO8ixhdjiVTRMoWHnw1CiXHUwMDAzXHUwMDFhXkJo4lx1MDAwMUcp6lx1MDAwMlx1MDAxNFx1MDAwZVx1MDAwNNi2VMGPKuDS4Caty29hZIvSW0mEscTmal+a3SbR1fnRtCl7ncmHcO9Q7/928M5/5tmtQOhcdTAwMTAuXGJcdTAwMTHAKS8kVtl8Yz3AXHUwMDE3/FpIh8W2XCLXt1x1MDAxZGctSmzhXHUwMDEwnjtKkFx1MDAwNHFcYuiX+0We2Wp4XHUwMDA3+jVH1EJcXFwi91x1MDAxMlskXHUwMDAyXHUwMDFjYZ7mXCL+eyq3XFyqUZzuh5+dXHUwMDFhXHUwMDE16l0kLHAszlx1MDAwNCVGl1x1MDAwNm35wzByXHUwMDFhzPP8zFx1MDAwM6CGqT8p2KRrfVx1MDAxNYX9OCtcdTAwMDKDXtlJ0rDjR/PudDTOezvg4qNonNzfpNEk7IexXHUwMDFmXHUwMDFk1HJ0leb2/NDSY1x1MDAwNZNJgqxcZoXr2Vx1MDAwN5154Vx1MDAxObQ0rNqcXHUwMDE3q1x1MDAxMvFfK716sUo+paR5/uYomL57/fqc7150WXr4xGfQ3ZErK57yXHUwMDEwWlx1MDAxMeVcdKtcdTAwMTlFMo1gU61XkWgjP0OmiFTD0OKJ9VOeUFNCXHUwMDAxXHUwMDFjyihCXYhcdTAwMTR8nlx1MDAwZVx1MDAxNopVpPxcdTAwMWOhkFx1MDAxMC1cdTAwMThlNFx1MDAxZnJ3XHUwMDFhI6lCai9ZvU8/d2f9l5ebTHpCKVx1MDAxNIBAZVdcbtpSxlrpXV7NOXx3Z1x1MDAxM1xu5ZzLSWmwQcs5cNVcXJaR1PBcdTAwMDNcdTAwMDL7c5VcdTAwMDdHRadcdTAwMWZFbbFF1y2X8kdJt7zWf0wutPhcdTAwMTWe4dXmOXxqVEVcYrF69Wxo+5VcdTAwMWR9/nBNPr++2Dr0t1x1MDAwZi7Cm1fs2cMnXG5PZCOaSFx1MDAwNFdUM8qyXHUwMDFhXHUwMDAwhZ4lyk+UsyRPap8hgDL0I3nLRfyJnz/x8yd+/ij8RLG9XGI/mdaM6MIx0VL4PL64atxsT3/feNvf3uy923w92jLJ31VMfid2OjVcdTAwMWHGiDScIFx1MDAxObe8XFxOMi49qdFKpGKAh8JNhCp2doKu6PpcdTAwMGZjJ293bLdd86KEaOdcZlx1MDAxY/UqXHUwMDExWop8MXPozOW+ey/CIbfiXHUwMDBi3ov8hMp/XHUwMDE2Klx1MDAxNym0MvuR3ktcdTAwMTZ6r+WUXHUwMDExzvJcdTAwMDC7zHn1yU4v2D377dO020GhNXw1OD06eO7Xl4RcdTAwMTBcdTAwMWXhiFx1MDAxZNhvQDq577yWKS01p+6s3JiKYE/jvEiqUKByQ1x1MDAxOdMoY1xyqTlcZrrnvdZag2JR/yxcdTAwMTSfofcu1mhl+kruu/Ag19Bqa/GeXHUwMDAyUvZvuKbApDLv3+5+UFx0vzlgw5uNZIs++9uHQlx1MDAxME/CNd3rK2a4JOV7OFZnsVlcdTAwMTjOrETO9mPKXHUwMDE2ST3CJLdwhbxumjsuM8JTXFxJIzVDYcLvXcBxN3Mgnf5Hw7B7LZqjzvdcdTAwMWTPokpEzMhcdTAwMGVoXHUwMDE5SkVleWnUXHUwMDAz57NfWvHaWmt9clx1MDAxOVx1MDAwNS18z766hrBcdTAwMGLZw15cdTAwMThMXFxzXHUwMDBi45O0tf5y1u153t04qFx1MDAxM2iRXHUwMDE05rrW//ZGo9m87OFloaftf6WYPZR7Pt/14GHWcev+3L6cXHQ59NPOIEhKct5x+vOO0FxcXG7Xi732ZzTPhvaqwG4tY5Nkayal1lGvl1x1MDAwNOn99oz5eVx1MDAxNMT9dOB6+V3n7ezhr/LeoFx1MDAxNT9/78n3T13+PbpcXOWdXHUwMDAyM+zBwPLgXHUwMDE1T8YkqTbnt9st6lx1MDAxOXe3ZuX4Yk7JeX+XTeP3ZLsx2lx1MDAxM83pyXDzuceXXHLlueRPKS5cdTAwMThcdTAwMTHGVO66U+Mu0GAvlFx1MDAxNYI+UNutelx1MDAxMa523DzQMOshSlx1MDAwYqZcdTAwMTDnhDQ1WVwi41x1MDAwNVx1MDAxMb5cdTAwMWWIaeQgbOH9zieML/OsJr/+Ntf+xk5fqN706vriXFzRXHUwMDBm715tRZ98Pl9iySizoDDvuL3LkJ4wXHUwMDBi/a5ki1ux+KRYMpudO65eLT28Ic/VXHUwMDFkaMVcdTAwMWTyK/szh3A3ZlZyiMdlXFxUeMiZmFVG12VcXIxBTMigiTFEyMJthruTXHUwMDBmXHUwMDE01lx1MDAwMulcdTAwMTj90S7xo9+If3/KtbffODw7lso/2el3mtH12fFNK1x1MDAxZbeHh33/5H3/dL9x4Te3xu2wXHUwMDE1N67OwsZVp3k0PT35cNWJ3/e7rbg5XHUwMDE4dDdcdTAwMWLTs1x1MDAxM/e7XHUwMDFiteNcdTAwMGafW/FO803/rLlFTo/PkjbvXFzuvLGt+Prs5O2g27TTzrQx6IB+t9mKXHUwMDA30Vx1MDAwZca1XHUwMDE5XHUwMDFkn05cdTAwMWJh9zhKzkAzujzDt3bTXvhcdTAwMWNcXMCjXHUwMDA1c2hcXHeG9mP3eDd6t9kgneFWK77ssLOrzpAkO6C709y9aseteJe2j1x1MDAwZmf890X/qDlcdTAwMTi3W/Ew6bfZqaNcdTAwMTW198F/2jhqxVx1MDAwN1x1MDAxZqPTbOZW46rLjqZcdTAwMWRcdTAwMDZGUbt53T9lNm1cdTAwMWZvXZ5tgz9zcjqJ2/yIdLaPyE7zbFxumoyApowyvsd00Fx1MDAxZe71W7HbNf9YXndP9sDNOI6gudOUUbC9pzb7o/89eVx1MDAxYbYspFYzs1x1MDAxYZXXKLxG3TXKrlF1jaJr1Fxco+RcdTAwMWFcdTAwMTXXKLhGvTXKrVFtjWJr1Fqj1Fx1MDAxYZUuUOhKuVx1MDAxOJuV/S6gZbCw7o/H+ylAYf3uwvj6VVx1MDAxOFxcN1x1MDAxNt9cInrxNWq56Fx1MDAxMGT3zG9f3P5cdTAwMWaSIGByIn0=yara { "rule": { "identifier": "test", ... "strings": { "$foo": "foo", "$bar": "bar", "$baz": "baz" } }, "matches": { "$foo": [ { "data": "Zm9v", "base": 0, "offset": 0, "match_length": 3 } ], ... } } QSBUZW56aXIgcGlwZWx pbmUgaXMgYSBjaGFpbi BvZiBvcGVyYXRvcnMgd GhhdCByZXByZXNlbnRz IGEgZGF0YWZsb3cuIE9 wZXJhdG9ycyBhcmUgdG hlIGF0b21pYyBidWlsZ GluZyBibG9ja3MgdGhh dCBwcm9kdWNlLCB0cmF uc2Zvcm0sIG9yIGNvbn N1bWUgZGF0YS4gVGhpb msgb2YgdGhlbSBhcyBV TklYIG9yIFBvd2Vyc2h lbGwgY29tbWFuZHMgd2 hlcmUgb3V0cHV0IGZyb 20gb25lIGNvbW1hbmQg aXMgaW5wdXQgdG8gdGh lIG5leHQ6Cgo=
We modeled the operator after the official yara
command-line
utility to enable a
familiar experience for the command users. Similar to the official yara
command, the operator compiles the rules by default, unless you provide the
option -C,--compiled-rules
. To quote from the above link:
This is a security measure to prevent users from inadvertently using compiled
rules coming from a third-party. Using compiled rules from untrusted sources
can lead to the execution of malicious code in your computer.
The operator uses a YARA scanner under the hood that buffers blocks of bytes
incrementally. Even though the input arrives in non-contiguous blocks of
memories, the YARA scanner engine support matching across block boundaries. For
continuously running pipelines, use the --blockwise
option that considers each
block as a separate unit. Otherwise the scanner engine would simply accumulate
blocks but never trigger a scan.
-B|--blockwise
Match on every byte chunk instead of triggering a scan when the input exhausted.
This option makes sense for never-ending dataflows where each chunk of bytes
constitutes a self-contained unit, such as a single file.
-C|--compiled-rules
Interpret the rules as compiled.
When providing this flag, you must exactly provide one rule path as positional
argument.
-f|--fast-scan
Enable fast matching mode.
<rule>
The path to the YARA rule(s).
If the path is a directory, the operator attempts to recursively add all
contained files as YARA rules.
Examples The examples below show how you can scan a single file and how you can create a
simple rule scanning service.
Scan a file with a set of YARA rules:
load file --mmap evil.exe | yara rule.yara
Memory Mapping Optimization
The --mmap
flag is merely an optimization that constructs a single chunk of
bytes instead of a contiguous stream. Without --mmap
, the
file
loader generates a stream of byte chunks and
feeds them incrementally to the yara
operator. This also works, but
performance is better due to memory locality when using --mmap
.
Let's unpack a concrete example:
rule test {
meta:
string = "string meta data"
integer = 42
boolean = true
strings:
$foo = "foo"
$bar = "bar"
$baz = "baz"
condition:
($foo and $bar) or $baz
}
You can produce test matches by feeding bytes into the yara
operator:
echo 'foo bar' | tenzir 'load stdin | yara /tmp/test.yara'
You will get one yara.match
per matching rule:
{
"rule" : {
"identifier" : "test" ,
"namespace" : "default" ,
"tags" : [],
"meta" : {
"string" : "string meta data" ,
"integer" : 42 ,
"boolean" : true
},
"strings" : {
"$foo" : "foo" ,
"$bar" : "bar" ,
"$baz" : "baz"
}
},
"matches" : {
"$foo" : [
{
"data" : "Zm9v" ,
"base" : 0 ,
"offset" : 0 ,
"match_length" : 3
}
],
"$bar" : [
{
"data" : "YmFy" ,
"base" : 0 ,
"offset" : 4 ,
"match_length" : 3
}
]
}
}
Each match has a rule
field describing the rule and a matches
record
indexed by string identifier to report a list of matches per rule string.
Build a YARA scanning service Let's say you want to build a service that scans malware sample that you receive
over a Kafka topic malware
.
Launch the processing pipeline as follows:
load kafka --topic malware | yara --blockwise /path/to/rules
If you run this pipeline on the command line via tenzir <pipeline>
, you see
the matches arriving as JSON. You could also send the matches via the
fluent-bit
sink to Slack, Splunk, or any other
Fluent Bit output. For example, via Slack:
load kafka --topic malware | yara --blockwise /path/to/rules | fluent-bit slack webhook=<url>
This pipeline requires that every Kafka message is a self-contained malware
sample. Because the pipeline runs continuously, we supply the --blockwise
option so that the yara
triggers a scan for every Kafka message, as opposed to
accumulating all messages indefinitely and only initiating a scan when the input
exhausts.
You can now submit a malware sample by sending it to the malware
Kafka topic:
load file --mmap evil.exe | save kafka --topic malware
This pipeline loads the file evil.exe
as single blob and sends it to Kafka, at
topic malware
.