python
Executes Python code against each event of the input.
Synopsis
python [--requirements <string>] <code>
python [--requirements <string>] --file <path>
A Python 3 (>=3.10) interpreter must be present in the PATH
environment
variable of the tenzir
or tenzir-node
process.
Description
The python
operator executes user-provided Python code against each event of
the input.
By default, the Tenzir node executing the pipeline creates a virtual environment
into which the tenzir
Python package is installed. This behavior can be turned
off in the node configuration using the plugin.python.create-venvs
boolean
option.
The python
operator implementation applies the provided Python code to each
input row one bw one. We use
PyArrow to convert the input
values to native Python data types and back to the Tenzir data model after the
transformation.
--requirements <string>
The --requirements
flag can be used to pass additional package dependencies in
the pip format. When it is used, the argument is passed on to pip install
in a
dedicated virtual environment.
The string is passed verbatim to pip install
. To add multiple dependencies,
separate them with a space: --requirements "foo bar"
.
<code>
The provided Python code describes an event-for-event transformation, i.e., it is executed once for each input event and produces exactly output event.
An implicitly defined self
variable represents the event. Modify it to alter
the output of the operator. Fields of the event can be accessed with the dot
notation. For example, if the input event contains fields a
and b
then the
Python code can access and modify them using self.a
and self.b
. Similarly,
new fields are added by assigning to self.fieldname
and existing fields can be
removed by deleting them from self
. When new fields are added, it is required
that the new field has the same type for every row of the event.
--file <path>
Instead of providing the code inline, the --file
option allows for passing
a path to a file containing the code the operator executes per event.
Examples
Insert or modify the field x
and set it to "hello, world"
:
python 'self.x = "hello, world"'
Clear the contents of self
to remove the implicit input values from the
output:
python '
self.clear()
self.x = 23
'
Define a new field x
as the square root of the field y
, and remove y
from
the output:
python '
import math
self.x = math.sqrt(self.y)
del self.y
'
Make use of third party packages:
python --requirements "requests=^2.30" '
import requests
requests.post("http://imaginary.api/receive", data=self)
'