Enrich with Threat Intel
Tenzir has a powerful contextualization framework for real-time enrichment of a
dataflow with a lookup table, a GeoIP
database, or a custom plugin. The heart of the
framework is a context—a stateful object that can be
managed with the context
operator and used with
the enrich
operator.
Setup a context
Prior to enriching, you need to populate a context with data. First, let's a
create a context called threatfox
that uses a lookup
table, i.e., a key-value mapping where a key is
used to perform the context lookup and the value can be any structured
additional data.
context create threatfox lookup-table
This yields the following pipeline output:
{
"num_entries": 0,
"name": "threatfox"
}
After creating a context, we load data into the context. In our example, we load data from the ThreatFox API:
from https://threatfox-api.abuse.ch/api/v1/ query=get_iocs days:=1
| yield data[]
| where ioc_type == "domain"
| context update threatfox --key ioc
Example data for context updating
If we replace the context
operator in the above pipeline with head 5
, we get
output similar to the following, depending on the current state of the API:
{
"id": "1213056",
"ioc": "deletefateoow.pw",
"threat_type": "botnet_cc",
"threat_type_desc": "Indicator that identifies a botnet command&control server (C&C)",
"ioc_type": "domain",
"ioc_type_desc": "Domain that is used for botnet Command&control (C&C)",
"malware": "win.lumma",
"malware_printable": "Lumma Stealer",
"malware_alias": "LummaC2 Stealer",
"malware_malpedia": "https://malpedia.caad.fkie.fraunhofer.de/details/win.lumma",
"confidence_level": 75,
"first_seen": "2023-12-15 15:31:00 UTC",
"last_seen": null,
"reference": "",
"reporter": "stoerchl",
"tags": [
"LummaStealer"
]
}
{
"id": "1213057",
"ioc": "perceivedomerusp.pw",
"threat_type": "botnet_cc",
"threat_type_desc": "Indicator that identifies a botnet command&control server (C&C)",
"ioc_type": "domain",
"ioc_type_desc": "Domain that is used for botnet Command&control (C&C)",
"malware": "win.lumma",
"malware_printable": "Lumma Stealer",
"malware_alias": "LummaC2 Stealer",
"malware_malpedia": "https://malpedia.caad.fkie.fraunhofer.de/details/win.lumma",
"confidence_level": 75,
"first_seen": "2023-12-15 15:31:00 UTC",
"last_seen": null,
"reference": "",
"reporter": "stoerchl",
"tags": [
"LummaStealer"
]
}
{
"id": "1213058",
"ioc": "showerreigerniop.pw",
"threat_type": "botnet_cc",
"threat_type_desc": "Indicator that identifies a botnet command&control server (C&C)",
"ioc_type": "domain",
"ioc_type_desc": "Domain that is used for botnet Command&control (C&C)",
"malware": "win.lumma",
"malware_printable": "Lumma Stealer",
"malware_alias": "LummaC2 Stealer",
"malware_malpedia": "https://malpedia.caad.fkie.fraunhofer.de/details/win.lumma",
"confidence_level": 75,
"first_seen": "2023-12-15 15:31:00 UTC",
"last_seen": null,
"reference": "",
"reporter": "stoerchl",
"tags": [
"LummaStealer"
]
}
{
"id": "1213059",
"ioc": "fortunedomerussea.pw",
"threat_type": "botnet_cc",
"threat_type_desc": "Indicator that identifies a botnet command&control server (C&C)",
"ioc_type": "domain",
"ioc_type_desc": "Domain that is used for botnet Command&control (C&C)",
"malware": "win.lumma",
"malware_printable": "Lumma Stealer",
"malware_alias": "LummaC2 Stealer",
"malware_malpedia": "https://malpedia.caad.fkie.fraunhofer.de/details/win.lumma",
"confidence_level": 75,
"first_seen": "2023-12-15 15:31:00 UTC",
"last_seen": null,
"reference": "",
"reporter": "stoerchl",
"tags": [
"LummaStealer"
]
}
{
"id": "1213060",
"ioc": "offerdelicateros.pw",
"threat_type": "botnet_cc",
"threat_type_desc": "Indicator that identifies a botnet command&control server (C&C)",
"ioc_type": "domain",
"ioc_type_desc": "Domain that is used for botnet Command&control (C&C)",
"malware": "win.lumma",
"malware_printable": "Lumma Stealer",
"malware_alias": "LummaC2 Stealer",
"malware_malpedia": "https://malpedia.caad.fkie.fraunhofer.de/details/win.lumma",
"confidence_level": 75,
"first_seen": "2023-12-15 15:31:00 UTC",
"last_seen": null,
"reference": "",
"reporter": "stoerchl",
"tags": [
"LummaStealer"
]
}
The pipeline context update
may yield:
{
"num_entries": 57,
"name": "threatfox"
}
That is, 57 entries have been added successfully to the threatfox
context.
Enrich with a context
Now that we loaded IoCs into the context, we can enrich with it in other pipelines. Since we previously imported only domains, we would look for fields in the data of that type.
The following pipeline subscribes to the import feed of all data arriving at the
node via export --live
and applies the threatfox
context to Suricata DNS
requests in field dns.rrname
via enrich
.
export --live
| where #schema == "suricata.dns"
| enrich threatfox --field dns.rrname
Here is a sample of an event that the above pipeline yields:
{
"timestamp": "2021-11-17T16:57:42.389824",
"flow_id": 1542499730911936,
"pcap_cnt": 3167,
"vlan": null,
"in_iface": null,
"src_ip": "45.85.90.164",
"src_port": 56462,
"dest_ip": "198.71.247.91",
"dest_port": 53,
"proto": "UDP",
"event_type": "dns",
"community_id": null,
"dns": {
"version": null,
"type": "query",
"id": 1,
"flags": null,
"qr": null,
"rd": null,
"ra": null,
"aa": null,
"tc": null,
"rrname": "bza.fartit.com",
"rrtype": "RRSIG",
"rcode": null,
"ttl": null,
"tx_id": 0,
"grouped": null,
"answers": null
},
"threatfox": {
"key": "bza.fartit.com",
"context": {
"id": "1209087",
"ioc": "bza.fartit.com",
"threat_type": "payload_delivery",
"threat_type_desc": "Indicator that identifies a malware distribution server (payload delivery)",
"ioc_type": "domain",
"ioc_type_desc": "Domain name that delivers a malware payload",
"malware": "apk.irata",
"malware_printable": "IRATA",
"malware_alias": null,
"malware_malpedia": "https://malpedia.caad.fkie.fraunhofer.de/details/apk.irata",
"confidence_level": 100,
"first_seen": "2023-12-03 14:05:20 UTC",
"last_seen": null,
"reference": "",
"reporter": "onecert_ir",
"tags": [
"irata"
]
},
"timestamp": "2023-12-04T13:52:49.043157"
}
}
The sub-record threatfox
holds the enrichment details. The field key
contains the matching key. The field context
is the row from the lookup table
at key bza.fartit.com
. The field timestamp
is the time when the enrichment
occurred.
For a more comprehensive discussion of contextualization, check out our blog post Contextualization Made Simple.