write_chunks

Converts each input event into a separate byte chunk.

write_chunks [field]

Description

The write_chunks operator reads a field from each input event and emits it as a separate byte chunk. The field must be of type blob.

Unlike write_all, which buffers all input and produces a single concatenated chunk at the end, write_chunks emits one chunk per event as data arrives.

This is the inverse of read_chunks.

`field = field (optional)`

The field to extract from each event. Defaults to data.

Examples

Write blob events to a file

from {data: b"hello"}, {data: b" "}, {data: b"world"}
to_file "output.bin" {
  write_chunks
}

Specify a custom field

from {payload: b"hello"}, {payload: b"world"}
to_file "output.bin" {
  write_chunks payload
}

Round-trip with read_chunks

from_file "input.bin" {
  read_chunks
}
to_file "output.bin" {
  write_chunks
}