Skip to main content
Version: Next

Enrich with Network Inventory

Tenzir's enrichment framework features lookup tables that you can use to enrich data in your pipelines. Lookup tables have a unique property that makes them attractive for tracking information associated with CIDR subnets: when you use subnet values as keys, you can probe the lookup table with ip values and will get a longest-prefix match.

To illustrate, consider this lookup table:

10.0.0.0/22 → Machines
10.0.0.0/24 → Servers
10.0.1.0/24 → Clients

When you have subnets as keys as above, you can query them with an IP address during enrichment. Say you want to enrich IP address 10.0.0.1. Since the longest (bitwise) prefix match is 10.0.0.0/24, you will get Servers as a result. The same goes for IP address 10.0.0.255, but 10.0.1.1 will yield Clients. The IP address 10.0.2.1 yields Machines, since it is neither in 10.0.0.0/24 nor 10.0.1.0/24, but 10.0.0.0/21. The IP adress 10.0.4.1 won't match at all, because it's not any of the three subnets.

Populate subnet mappings from a CSV file​

It's common to have Excel sheets or exported CSV files of inventory data. Let's consider this example:

subnet,owner,function
10.0.0.0/22,John,machines
10.0.0.0/24,Derek,servers
10.0.1.0/24,Peter,clients

First, create the context:

context::create_lookup_table "subnets"

Then populate it:

load_file "inventory.csv"
read_csv
context::update "subnets", key=subnet

Enrich IP addresses with the subnet table​

Now that we have a lookup table with subnet keys, we can enrich any data containing IP addresses with it. For example, let's consider this simplified Suricata flow record:

{
  "timestamp": "2021-11-17T13:32:43.237882",
  "src_ip": "10.0.0.1",
  "src_port": 54402,
  "dest_ip": "10.1.1.254",
  "dest_port": 53,
  "proto": "UDP",
  "event_type": "flow",
  "app_proto": "dns"
}

Let's use the enrich operator to add the subnet context to the two IP address fields:

load_file "/tmp/sample.json"
read_json
context::enrich "subnets", key=src_ip, into=src_ip_context
context::enrich "subnets", key=dest_ip, into=dest_ip_context

This yields the following output:

{
  "timestamp": "2021-11-17T13:32:43.237882",
  "src_ip": "10.0.0.1",
  "src_port": 54402,
  "dest_ip": "10.1.1.254",
  "dest_port": 53,
  "proto": "UDP",
  "event_type": "flow",
  "app_proto": "dns",
  "src_ip_context": {
    "subnet": "10.0.0.0/24",
    "owner": "Derek",
    "function": "servers"
  },
  "dest_ip_context": null
}

We have enriched all IP addresses in the flow event with the subnets context. Now go hunt down Derek!