Run pipelines You can run a pipeline in the
app or on the command line using the tenzir
binary.
In the app Run a pipeline by writing typing it in the editor and hitting the Run button.
The following invariants apply:
You must start with a source The browser is always the sink The diagram below illustrates these mechanics:
eyJ2ZXJzaW9uIjoiMSIsImVuY29kaW5nIjoiYnN0cmluZyIsImNvbXByZXNzZWQiOnRydWUsImVuY29kZWQiOiJ4nO1ba3PTOlx1MDAxM/7Or8j0/Up8tLrrfKNAoaXcmlx1MDAwMoV3znSc2ElM09i13SaB4b+flUPjS+KmpUlcdTAwMWI49UCaSLK0lvZ5dldaf3/UaGylk8jf+rux5Y877iDwYne09diWX/hxXHUwMDEyhEOsotnvJDyPO1nLfppGyd9//eVGkZPf5XTC0+md/sA/9Ydpgm3/j79cdTAwMWKN79lnYaxBMPSztllpYSTDqqVvwmE2qiBKcKUlmTVcYpJnOFLqe1jbdVx1MDAwN4mf13iZmG2xo87GR/v+3ri7d9xyXHUwMDAz1Zrko3aDwaCVTlx1MDAwNtNHcjv987ggU5LG4Yn/KfDSPtZDpXx2X1x1MDAxMuLD53fF4XmvP/STpHRPXHUwMDE4uZ0gndgykovvXHUwMDBle1lcdTAwMWZ5ydi2cDhcdTAwMDdFjaZagWCU81m17YBrcJjk2lx1MDAxMFwimJaaVVx1MDAwNHtcdTAwMWFcdTAwMGXC2Fxu9j+SXblobbdz0kP5hl7epuN73HPzNqOfj6tzmfp+0OunduS8LPGzOTeGXHUwMDAxSKXkrML2XHUwMDFm7XrZwv+TT3Tsnvq79o7h+WBQnKuh93OuSlx1MDAxNW1b8bygQ3lX55HnTlx1MDAxN1x1MDAxY6TWQnLOXGZcdTAwMTA6q0e9Oql2N1xiOydcdTAwMGJ0JEndON1cdTAwMGWGXjDsVW/xh15NzcBN0qfh6WmQolx1MDAxOO/CYJhWW2T9PonjcNT33bmHxp5r61wi211cdTAwMGVcdTAwMWF75d9cdTAwMWG5nmQ/Zt//eby8NV/aXFzesL0utafEIcVcdTAwMGJuNlx1MDAxY3OEoFxumFRao15cdTAwMTPwmyBu9IBcdTAwMDV5XHUwMDFmXHUwMDE1//54fDP+YVLU8Vx1MDAwZjBukJ5kQd2XXHUwMDEx0KtDczHZf0vMWfpNk/FepGBbbDxcdTAwMDFJcFx1MDAwNDHAKONEXGLCc3RdXHUwMDEykGLITkg+QipT4KeVMlx1MDAxMFxuXHUwMDAxhDFUXHUwMDBizVxyiHxZriAkLilcdTAwMDdjTC7xXHUwMDAzIW1cZiHRXG5j3DeB1SlYJlx1MDAxZHNcdTAwMTQxXHUwMDA0iJaGSi6B/baEpmm19JLQXHUwMDE0XHUwMDAyXFxxQnJcdTAwMDAv4zP33auLY//w+MUz6L2Uk/H+/lx1MDAxN042nc8o8lx1MDAxOSdAiVKMXHUwMDAwPnDO35d8Jo1mXHUwMDFj+Y7SUvW981x1MDAxOWjggFx1MDAxZa/WXHUwMDBmhPZAaEva/0dcYk1oXi2deWhcdTAwMDBcZlDKXHUwMDFjW8tcYu34xftDXHUwMDExf1x1MDAxZfU+RLo3jujbJ093tzed0DilXHUwMDBl01x1MDAwNlx1MDAwMN1cdTAwMWShRNU9I45cdTAwMTbarjVDR6jAM7/CZl1P+cBcdTAwMTfFh2V/P49CZ2zGjKM4k1x1MDAxOKOCUZqIgps4JTf0Lo2iRNxP9KhcYpWMXHUwMDEwLVx1MDAxMC1cdTAwMGbcNtdcdTAwMWG0QzPfn1wi91Ogpng3VJd2WW9NXGZcdTAwMTk4N0IpbfA/Mlx1MDAxMS/2N6cqy/pTZelcdTAwMTi9qje6tDflaM1BYvBAkFx1MDAxZKFEnIBoXHUwMDAydFx1MDAxYjSg0KgrTK+V9VJ/nC5iPaCyWjpjPVx1MDAxNFBKfVx1MDAwMzeunYjPSXhcdTAwMTB1e93ghF6MXHUwMDA24+3meMWs54VWxVfqx3HjSFxcXHUwMDAxlVPalPC0Q4RcdTAwMTaGqdtxXHUwMDFkXGLXbXfnuVx1MDAwZaSDLpDRODS6kviZ88WM7Ghh6Cm7oZlC/UOV+i09t244TFvBN7tcYrmXk5XuuKfBYFJaxExncVx1MDAwNlx1MDAxZMfZKlx1MDAxNT5cdTAwMTlcdTAwMDS9YWbL/W5Zq9Og41x1MDAwZWbVaVx1MDAxOOW1XHUwMDFkXHUwMDFjxEXbXHUwMDFmz09KXHUwMDE4XHUwMDA3vWDoXHUwMDBlXHUwMDBlXHUwMDE3XHJo/YWXl+tcdTAwMDFOYUXabuJn3oTFyq+hXHUwMDBmaneFmDBaUENypViGPnIo9rxcdTAwMTZRXHUwMDAzOHv+lG/3veig29l49DWhXHUwMDEwnmS4I8phhlx1MDAwYvS4boc7TTtcdTAwMDb8edxJ7oCyW8xcdTAwMTJpmS6Il+ZBx1xiN7hcdTAwMTT/JdC1pocyd4q76phrhVx1MDAxZYVq6czwSUBcdTAwMGKvXGK/vuU7ir+STyNGeXNcZqSfXHUwMDFlfVx1MDAxZVxy2M7GY0+giWNGc1Y1fYU7V4k8Jlx1MDAxY7RdRnKBXHUwMDFmpGRYr1x1MDAwMFx1MDAxZjCUXHUwMDExXHUwMDFkssLZ0ZrQN/Opvs++zVY42n55+qT1hnbJp7Pw29GxXHUwMDFh73eOZo9YUjnXetJbs5pcdTAwMWaXXHUwMDBl2uZg2/Z/t8gujbhOXFxTqWpxbXC2XHUwMDE1NVx1MDAwNYOzXGbXXHUwMDFmPEZcIvfLKGieXHUwMDFkvjt9O1x1MDAxOFx1MDAwNWmkNlx1MDAxZdcgyyaV6fXgXHUwMDE5x3GEQlx1MDAwMjGUT83pdfAsmZSSXHUwMDEwtfbDlKvgLDr8Xe94pznpe/7gRWswkfHE/KZwPozdYdJccuNTN0VtT+5cdTAwMTbZtYOvXHUwMDAy5NOpX4ByXlx1MDAxZrZye5ZKdGH7alx1MDAxOciv1oVNXHUwMDA1Oa36zXQ9IC90e8WJgpJcdTAwMDKj2Fx1MDAxNfjHOVYvteByn+XHXHUwMDFku82VnbiCYD/TkHavYVwiMvR2zq2UTeKgalwiU6JYhqBcdTAwMDekhSo067mRlVrMPe2d7fptpXEw1bVZi5Xu/TVXdrDxqPj3xuRBab2LQJVC40Rv4PrvxOQw/tLbXHUwMDFkh+3DYEd2z3n/IN549mhcbuWApFx1MDAxMs22XCJcdTAwMDR1cbZcdTAwMTE7JVx1MDAxMzBcdTAwMGWhlDDBXHUwMDE0XHUwMDA2vuqWW2B13KKZwzXhUi+KXHUwMDA0OHNcdTAwMTg3iitDXHRoUzg+nVJcdTAwMGVcdTAwMDXgRHOgK4hcdTAwMGLWwjr54cXvtv+/biZcdTAwMTDUUcyGnZdpXcW7jWMw9GaCT5d9WV9cdTAwMGJ0yF5V7VlcdTAwMTl56NpEUlx1MDAxNFlcdKBwfc/j6qByU7lDKOlo5IbiXHUwMDA23ZQ0cFkxwFJGXHUwMDE5o1x1MDAxMVx1MDAwMWvhXGYjcWGFPZxhUoOGRcyhXHUwMDFjQ61cdTAwMTOI/+zW+lx1MDAxY3VgXHUwMDExaC4lW0FcZrKx1FHnsFxcvVdVcFgyf8Vm4klcdTAwMDKALkthQyz3V1x1MDAwMCNAjZrwXHUwMDA3uy3C5i1cbiZcctdTjStcdTAwMWQyanuKQ1x1MDAxOfupa8Is7a9Wg+01p7s3JK7afC1FqqXF/U4uXHUwMDE4uf62yPPU1e+3o94xv6BcdTAwMWbEwcdXTXOw6ohp5elccjaPRVx1MDAxYW1AMWFcdTAwMDBY7lx1MDAwNF7mN1x1MDAxOClcdTAwMDRHbWdK3s7fWWmyXHUwMDE2+jmKMyvU7cnqdqdcdTAwMGa/r0PzkKyVSffHJGtxVnt6w1x1MDAwNVoqdpNcYu7p14TF7W/J3vP43Gu/XHUwMDBl/P7Zy1Vv8q6czVx1MDAxOCeOUkxcdTAwMWJl/bCiUcqz6bVcdTAwMTDGXHUwMDEwhmvFTUWwVb3OU3k9Y57PmHYocG5jTDBGwIL9YEHRXG4quPfk+lx1MDAwN3qbb1xyxqHohlx1MDAxMIrxXHUwMDE1R0opbllNs7lKi7vU96koLTPlXHUwMDE0qzllWdahqMgnb9edqnSnb/W0982e9dvn9UlfUnBcdTAwMDLC6Os7g88uPlx1MDAxZGx/22uRj71m++W7oey4r71cdTAwMWH6rNDgfZEncaTkmlx1MDAxM2r3IKRSkHdjO8ApcIRh1GiMfoSk9e9CdonuLM11ddu43Fx1MDAwYk7DjXFcdTAwMThcdTAwMTM42ShcdTAwMDO1r1x0zLNnWVx1MDAwZeTJuUiWXHUwMDE4g7auaO7+vEh2faSKXHUwMDBiVtirrVx1MDAwNpalypXSav3a26s5t+w3XHUwMDA0fm3SXHUwMDBiqX9cdTAwMGJRXHUwMDBizYjA0PPawD/6XG5cdTAwMWaS3aS9M26lb1+395vbr4dcdTAwMTe/XHUwMDAyfFopX2NcZqgwXHUwMDA2ZFx1MDAxOOpiwMsxrCrvelx1MDAwYlF/hHZcdTAwMWKkUyps6rDWgN9sSmlcdTAwMGXW/Jx8/i1otI5cdTAwMTiIrmKP6lx1MDAxZfyiwkk2yFLpXHUwMDE1J9mf+m7amITnjVFcdTAwMWOkfiNcdTAwMTg20r7f8L0gXHJcdTAwMGJILFx1MDAxZGt38Cn8uKTsqznYvp4s1zvl5r+EVlqPVi6J5vbls2uDlX3gY/pKm07wgo7fx+mbXHUwMDEz8lFutpVcdTAwMTZcdTAwMTWzzFxyhlx1MDAxYoJk2423w2nXNdxbgFPtwKKEUOtcdTAwMDTPXHUwMDE5XkqoXHUwMDAwXGai156Vdlx1MDAxNTxcdTAwMDVhRlx1MDAxNJVxhfDMYX9cdM8nUXTXKCxccllcdTAwMDXbQqxNXWqbpZNN3JZcdTAwMWJFrVx1MDAxNKdt5uBsXVx1MDAwNP5oe6Fa2Mtm+mRItZjwM7/ox6NcdTAwMWb/XHUwMDAymsxcdTAwMTjwIn0=... Source Sink Transformations What you write in the editor App
For example, write version
and click Run to
see a single event arrive.
On the command line On the command line, run tenzir <pipeline>
where <pipeline>
is the
definition of the pipeline.
If the pipeline expects events as its input, an implicit load - | read json
will be prepended. If it expects bytes instead, only load -
is prepended.
Likewise, if the pipeline outputs events, an implicit write json | save -
will
be appended. If it outputs bytes instead, only save -
is appended.
The diagram below illustrates these mechanics:
eyJ2ZXJzaW9uIjoiMSIsImVuY29kaW5nIjoiYnN0cmluZyIsImNvbXByZXNzZWQiOnRydWUsImVuY29kZWQiOiJ4nO1ba1faSlx1MDAxN/7eX8Hy/VrmzP1yvlx1MDAxNVtvtdUj9vqus1xcXHUwMDAxXHUwMDAyRCPBJFxu2NX/fvZcdTAwMDQlXHUwMDE3iKCC0taspeLMZGYns59nX2bz41Wlslx1MDAxMY/67sbflVxyd9h0fK9cdTAwMTU6g43Xtv3KXHIjL+hBXHUwMDE3Tf6PgsuwmYzsxnE/+vuvv5x+XHUwMDFmpXehZnA+vtP13XO3XHUwMDE3RzD2//B/pfIj+Z1Zy/d6bjI2aU1XYlhcdTAwMTRbP1x1MDAwNr1kVYJcclOcsnSAXHUwMDE3vYWVYrdcdTAwMDW9bceP3LSnlYjZXHUwMDEwW+pi+HXf3Vx1MDAxYrb3TuqOp+qjdNW25/v1eOSPXHUwMDFmyWl2L8OMTFFcdTAwMWNcdTAwMDZn7lx1MDAxN69cdTAwMTV37eKF9sl9UVx1MDAwMFx1MDAwZp/eXHUwMDE1XHUwMDA2l51uz42i3D1B32l68Sh5XGI8aXV6nWSOtGVoRyBGXHUwMDE51VRcYqxccteKyUm3nYBrXHUwMDE4ILk2XHUwMDE4XHUwMDBipqVmXHUwMDA1wTZcdTAwMDM/XGKtYP/DyZWK1nCaZ1x1MDAxZJCv10rHtNuO4Vx1MDAxOfFcdTAwMDc3j6tTmbqu1+nGduW0LXKTd25cZiNEKpVKaOfv77aSjf83fdGhc+7u2jt6l76ffVe91s27ynU0bMe7jFx1MDAwZaVTXfZbznjDicJUMow1wVpN+kGvzorT+UHzbIaORLFcdTAwMTPGNa/X8nqd4i1ur1XS4ztRvFx1MDAxOZyfezGIcVx1MDAxOHi9uDhcIpn3TVx1MDAxOFx1MDAwNoOu60w9NMxcXNrXt9OloLFX+qmS6knyz+Tzv6/nj+Zzh8t7jte58Vx1MDAxNCOcvcj9lmNIXGKqXGKTSmvQa0zcKlx1MDAxMfd6wIy8r7J/f76+J/8oXFxsnfBcdTAwMGYlQEGG6Vx1MDAxNNfzXGLo/bG5XHUwMDFh7Vx1MDAxZmBzXHUwMDExX2s83OsrUlx1MDAxM2tPQJIggVxyXHUwMDAxXHUwMDFh4lx1MDAxOFiI0ylcdTAwMDJSTFNccuQjpDKcP4qBmm6Lt5xcdTAwMTlcZlx1MDAwNEJcdTAwMTDMXHUwMDE4qIXmhoiU9e8gJC4pJ8aYVOJnICSQN6NcIi+EdHvRXHUwMDAyYzw3gZUpWFwiXHUwMDFkQ1xuXHUwMDFiXGamRVx1MDAxYSq5JOyXJbSMcSxcdTAwMTBcdTAwMWFcdTAwMDPDiVx1MDAxNU/759GZc/j+6sQ9Ptl+Szo7cjTc3//O8brTXHUwMDE5XHUwMDA1OuOYUKxcdTAwMTTDRHE87U9Jo1x1MDAxObdeXHUwMDE3zXU/O51cdTAwMTFNONFcdTAwMTJr/cJnL3w2Z/xcdTAwMWbCZ8KQYmtcdTAwMWEgalxycVx1MDAxMTzmwox2sv3PsVxiv1xyOp/6ujPs04M3m7u1dWc0TimC/TWUXGKuXHUwMDA0p6ZAaFx1MDAxOGmh7Wtg4Fx0ZZhmuVx1MDAwMWLe4U/D0Fx0nzGDXHUwMDE0Z1x1MDAxMoJUYpTGXCLjJ47pXHLcS6MoXHUwMDE2L+HjXHUwMDFhslx1MDAxYtGIJs4/JVx1MDAxOH5SJUs6i1s7b7YqxFxmnFx1MDAxYqGUNvBcdTAwMDNcXMSz802pyrz5VF46Ru+ajc6dTSGtOZFcdTAwMTA9YGlhlXtWQFx1MDAxM1x1MDAwMcdcdTAwMDFcdTAwMTTFcOBWplfKe7E7jGfxXHUwMDFlYazYest7XHUwMDEwilx1MDAxMSpIao/m0l4jXHUwMDEy36LgqN/utL0zejXwh7XqcMm011xurIov1ZPjXHUwMDA2SdhcdTAwMDGVUtqY8DTCQlx1MDAwYsPU47iOXGLHabSnuY5IxIFKNSxccs4k/E7dsVx02dHM0mN2XHUwMDEzXHUwMDFhzKxcdTAwMDGVekZ2I0SBuFx1MDAwZmG3dtCL6951ol4417rlnHv+KLeJic7CXHUwMDFiRFxibeRcdTAwMWHf+F6nl1hzt53X6thrOv6kO1x1MDAwZfppb1x1MDAxM1x1MDAxNnHA+ofTLyVcYr2O13P841lcdTAwMGJaj2Hndj9cYsrsSMOJ3MSfsC71w9BHS9FcdTAwMDexg4BXrdKcwzz04WOx16pj5ZOLd5u81m31j9rNtUdflWRcdTAwMDKUXHUwMDA0d1ghZrhghD5cdTAwMGV3mjZccnGncSc5XCLK5pgl0DKdXHUwMDExMU2DjmFuXGbWf1x1MDAxMujq41OZJ8Vdcc3VQk9cdTAwMTdbU4dcdTAwMWZzXHUwMDAxzJ9cdTAwMTkxXHUwMDBme1/DU/xlwCivXHUwMDBlXHTuxl+/XHJ8trX22Fx1MDAxM2DimNGcXHUwMDE1TV/mzmVcIo9cdFx1MDAwNLbLSC7gXHUwMDE3zlx1MDAxOdY7wFx1MDAwNy6K0eCQZVx1MDAwZY9WhL6JT/Vj8mmyw/3azvmb+kfaxl8uguuvJ2q43/w6ecScyjnWk96Y9Py8ddDWXHUwMDA323b+p0V2bsVV4lx1MDAxYUxmXHUwMDE5rrVkhlx1MDAwMUZcdTAwMTeH9adcdTAwMTbDfef7wKteXHUwMDFjXHUwMDFmnlx1MDAxZvhcdTAwMDMv7qu1hzWReYvK9GrgXGbrIKGAP1xm5WNrulxinCWTUuKsY/NcZmhcdTAwMTZNftg52aqOui3X3677I1x1MDAxOY7MUtFsstq+UjRcdTAwMWaHTi9qXHUwMDA34blcdTAwMTODskdPXHUwMDBi7NLFl4Hx8aufXHUwMDAxcsHLs3VcdTAwMTBH2fSPWTxbd7cyrCvKadFvpitCOUMm74xPwC1cYip4XHUwMDBmN1x1MDAxOFdSQEy7XHUwMDA0bzmF7q1S3GZdfj7IicbgyVx1MDAxM5ZV0lx1MDAwN+blMoLdVCXtLmAxXHUwMDEyMDcvrZRVXGZRXGKEXHUwMDE1tqpcdTAwMDFcdTAwMDI964apzLCO07dSi6mnfbJcdTAwMWPgRlx1MDAxY3pjzZuMWGomsEow4jSbdrFcdTAwMTfRyJAkLpt3P0M5rbTXRFx1MDAxZtN7X2X/3ptnJOHF1skpJ8SDWlx1MDAxYbz4MedWiI/D753dYdA49rZk+5J3j8K1p5mqUIhIwFx1MDAwZZFcbmMjdeFYgCgksORcdTAwMDQzXHUwMDA1b0Q9MldWRkKKI2oorD4rZODcSki5Mlx1MDAxNFx1MDAxM20yJ61jNqJcdTAwMDQ0SvNsJnN9XGKJcpopdfnVXHUwMDBlXG5WTVx1MDAxMoJcIsUsom9cdTAwMGLAsndcdTAwMWJkMOdM8PG2z5trhlx1MDAwZdmrqD3L4lx1MDAwZapLXHUwMDEzXGaaa82xYCmU5nHH3dHnunKHUFx1MDAxMmmKc87DmDRgW7WiyihjNDNFkZbDXHUwMDE5RlwixoU9xWFSXHUwMDEzTWYxh0JcdTAwMDZ6sS2GsDn4KeqAJqK5lGxcdNHK0qljWVx1MDAxNVx1MDAxNGW+zN1JrYwvg1x1MDAwMVjM1uxJiH3AJmKeOYa7dWVcYsSK9kDpN/ZohK1wXHUwMDE0TFx1MDAxYa7HXHUwMDFhlzuN1Pa4hzJ2o2vCzJ2vVIPtNaW79ySu8squ0jNcdE5ccmEqc2Yxj7bexY7+p9bvnPAr+klcdTAwMWN9fl81R8uOrJZeXHUwMDA3oVx04oJwuJJyXHUwMDA3U6xTxchIITgoO4M451HUVVpcdTAwMDbxkLIucHNcdTAwMTRnVqjHc9VL4cOyyeGlrOs+XHUwMDBmvKyyLnBcdTAwMGbLyIwwXHUwMDFiXHUwMDE1QGSzeKJo8zRiYeM62ntcdTAwMTdetlx1MDAxYVx1MDAxZjy3e7Gz7HTw0umMcYyUYtoo64dljVJad6+FMFx1MDAwNjPYLG5cboKtpq4rfZS0rksjXG6sa2NMYowgMzLHgoJcdTAwMTVU5FnL8F/4bTZ8iUFcdTAwMTSwXHUwMDA0L0krXHUwMDBlnFwiVPZ2Utzcub5PQWmZyddiTSnLvFx0RUE++bjpVGE6/ainfW76LFxyYokp/94kOJ4qm0ydx55vr75cdTAwMWPVrvfq+HOn2tg57Mmm86FVwp5cdTAwMDVcdTAwMTZ8tu8sgdZcdTAwMTjOhMKSU9BcdTAwMTmTL/JcdTAwMTe8POXexro5ly2dXHUwMDA27O+MlLvSSFx1MDAxN0tgJkQ5Xc+PjaE8l0f6/YLU1fElbE0mXHJbjFx1MDAxOXOdy2XMXHUwMDE5u1xcuHEh8JaXdtJi66S4XGbiSVx1MDAxYqem6jy3wOWUfIp2o8bWsFx1MDAxZVx1MDAxZnxo7FdrXHUwMDFmeldcdTAwMGZcdTAwMDEvLbSvXHUwMDEwvFx1MDAxOCOOZVJcdTAwMTnMlTRcdTAwMDXHR8jVgJeCaWAmsYB4NoDpXHUwMDE0glxy2DeIJZeRZXpcXH3ZY0+ticy13nFq/aXrxJVRcFlcdTAwMDE1tifR/sh+citBr1x1MDAxMnfdyub+bslBdlx1MDAxM57FXHJzXG6/nKPs+0i02Ok2f1x1MDAxMG5lRj+KNtfALlx1MDAxMSUy681cdTAwMDMua/Vrp9+va6RcdTAwMTPuXHUwMDEw/nF7e3Pvc1ll2rpYXYYwXHUwMDE4XVx0IShOykpSfztcdTAwMDEuI1xiQlx1MDAwMakkXHUwMDAxkyeFLlxidk9cdTAwMTiXRCxColnYtW7ulP2lXHUwMDA05JWGpHL+St+nW1x1MDAwML6pk3dcdTAwMGJfP3BalepTQ7S4alx1MDAxMYYzUShcdTAwMWWGwjRCmEKholxu3HKsXHUwMDE2R+HpVq9cdTAwMTW5h9XN91x1MDAxMd3yyfvj3ZF6t94oVFxuKWMzy3yMwsJcdTAwMTGOXHUwMDAwkEJoIyAgT0BY/nX9x6BQczSzSnRcdTAwMTZcZsGXUlx1MDAxYYLFP1xihSE4qZXTXGJcdTAwMTT2iYE4Y+FcdTAwMTViUdPSmk5CiFx1MDAwNs/OyHtYRJ9/voxcdTAwMWH+XHUwMDFiXFw721x1MDAxZVx1MDAwNIPrbXrkrTdcdTAwMTZcdTAwMTljXGKMnYErXHUwMDAxYzpLXHUwMDFli3KFXHUwMDA20TC0KFx1MDAxMonUilLyvN/AfFokXHUwMDBlQi92n1x1MDAwNYqzVl6lXTSm2JraRcK0PS1fvCaqfi5cdTAwMDbf8UHf36l5zavNt/6eOnu73ljkjFwiYVx1MDAxOCAtMYzMXHUwMDE0wsqJd8pcdTAwMTLDqFxugj29d2pz1kbgzL799miMnCv36b3T4qr3QeGrmzTRhtPv12N4eVx1MDAxYre5uI0rz1x1MDAxZNRm6oa9bGV6gmFcdTAwMGJcdTAwMTY3SeH9fPXzP+Kzx+AifQ==... Source Sink Transformations What you typically type on the CLI load - read json write json save -
For example, run tenzir 'version'
to
see a single event rendered as JSON:
{
"version" : "v4.6.4-155-g0b75e93026" ,
"major" : 4 ,
"minor" : 6 ,
"patch" : 4 ,
"tweak" : 155
}
You could also render it differently by passing a different
format to write
, or by
inferring the format from the file extension:
tenzir 'version | write csv'
tenzir 'version | to /tmp/version.ssv'
tenzir 'version | to /tmp/version.parquet'
Instead of passing the pipeline description to the tenzir
executable, you can
also load the definition from a file via -f
:
tenzir -f pipeline.tql
This will interpret the file contents as pipeline and run it.
As Code In addition to running pipelines interactively, you can also deploy pipelines as
code (PaC) . This infrastructure-as-code-like method differs from the app-based
deployment in two ways:
Pipelines deployed as code always start with the Tenzir node, ensuring
continuous operation. To safeguard them, deletion via the user interface is disallowed. Here's a an example of deploying a pipeline through your configuration:
<prefix>/etc/tenzir/tenzir.yaml tenzir :
pipelines :
# A unique identifier for the pipeline that's used for metrics, diagnostics,
# and API calls interacting with the pipeline.
suricata-over-tcp :
# An optional user-facing name for the pipeline. Defaults to the id.
name : Import Suricata from TCP
# An optional user-facing description of the pipeline.
description : |
Imports Suricata Eve JSON from the port 34343 over TCP.
# The definition of the pipeline. Configured pipelines that fail to start
# cause the node to fail to start.
definition : |
from tcp://0.0.0.0:34343 read suricata
| import
# Pipelines that encounter an error stop running and show an error state.
# This option causes pipelines to automatically restart when they
# encounter an error instead. The first restart happens immediately, and
# subsequent restarts after the configured delay, defaulting to 1 minute.
# The following values are valid for this option:
# - Omit the option, or set it to null or false to disable.
# - Set the option to true to enable with the default delay of 1 minute.
# - Set the option to a valid duration to enable with a custom delay.
restart-on-error : 1 minute
# Add a list of labels that are shown in the pipeline overview page at
# app.tenzir.com.
labels :
- Suricata
- Import
# Disable the pipeline.
disabled : false
# Pipelines that are unstoppable will run automatically and indefinitely.
# They are not able to pause or stop.
# If they do complete, they will end up in a failed state.
# If `restart-on-error` is enabled, they will restart after the specified
# duration.
unstoppable : true