Enhanced UDP Source

This release enhances UDP ingestion with the new from_udp operator that produces structured events with sender metadata. We also improved the execution model for every and cron subpipelines, added DNS lookup capabilities, and made the Syslog parser more flexible.

Download the release on GitHub.

Features

Receive UDP datagrams as events

The new from_udp operator receives UDP datagrams and outputs structured events containing both the data and peer information.

Unlike load_udp which outputs raw bytes, from_udp produces events with metadata about the sender, making it ideal for security monitoring and network analysis where knowing the source of each datagram is important.

Each received datagram becomes an event with this structure:

from_udp "0.0.0.0:1234"

{
  data: "Hello, UDP!\n",
  peer: {
    ip: 192.168.1.100,
    port: 54321,
  },
}

Enable hostname resolution for DNS lookups (disabled by default for performance):

from_udp "0.0.0.0:1234", resolve_hostnames=true

{
  data: "Hello, UDP!\n",
  peer: {
    ip: 192.168.1.100,
    port: 54321,
    hostname: "client.example.com",
  },
}

By @mavam in #5375.

Perform inline DNS lookups

The new dns_lookup operator enables DNS resolution for both IP addresses and domain names. It performs reverse PTR lookups for IP addresses and forward A/AAAA lookups for hostnames, returning structured results with hostnames or IP addresses with their types and TTLs.

Resolve a domain name to IP addresses:

from {
  host: "example.com"
}
dns_lookup host

{
  host: "example.com",
  dns_lookup: {
    records: [
      {
        address: 2600:1406:3a00:21::173e:2e65,
        type: "AAAA",
        ttl: 58s,
      },
      {
        address: 23.215.0.136,
        type: "A",
        ttl: 2.433333333333333min,
      },
      // ... more records
    ],
  },
}

Resolve an IP address to a hostname:

from {
  ip: 8.8.8.8
}
dns_lookup ip

{
  ip: 8.8.8.8,
  dns_lookup: {
    hostname: "dns.google",
  },
}

By @mavam, @IyeOnline in #5379.

contains_null(x:any)

We added a new contains_null function that checks if the input value contains any null values.

By @raxyte in #5419.

Context for assert operator

The assert operator now has a message option that can be used to provide context about the event failing the assertion.

By @raxyte in #5433.

More lenient RFC 3164 Syslog parsing

Our syslog parser now allows for a . character in the tag/app_name field and any character in the process_id field. This allows you to parse the log:

<21>Aug 18 12:00:00 hostname_redacted .NetRuntime[-]: content...

{
  facility: 2,
  severity: 5,
  timestamp: "Aug 18 12:00:00",
  hostname: "hostname_redacted",
  app_name: ".NetRuntime",
  process_id: "-",
  content: "content...",
}

By @IyeOnline in #5426.

Changes

every and cron subpipelines

We changed the execution model for every and cron subpipelines, resulting in:

• operators such as context::load now execute properly.
• subpipelines can contain both remote and local operators.
• subpipelines must not accept or output bytes.

By @raxyte in #5410.

Deprecation of split_at_null option of read_lines

The split_at_null option of the read_lines operator is now deprecated. Use read_delimited "\0" instead.

By @jachris in #5431.

Amazon Security Lake

We have made two convenience changes to the to_amazon_security_lake operator:

• The role parameter now defaults to the automatically generated role for the custom source in Security Lake. If you are using a different role, you can still specify it.
• The operator now uses UUIDv7 for the names of the files written into the Security Lake’s blob storage. Since UUIDv7 is time ordered, inspecting the files in the lake becomes slightly easier.

By @mavam, @IyeOnline in #5412.

Sorting Improvements

We have re-done the internals of the sort operator. You will now be able to more reliably sort events using lists or records as keys. Lists are compared lexicographically between their values, while records are compared by their sorted key-value pairs.

By @IyeOnline in #5425.

Bug Fixes

Rare crash in save_tcp operator

We fixed a rare shutdown crash in the save_tcp operator.

By @jachris in #5420.