Export from a node Exporting (or querying ) data can be done by running a
pipeline that begins with the
export
source. When managing a pipeline through
the app or the API, all pipeline operators run within the node. When using the
CLI, at least the export
operator runs within the node.
eyJ2ZXJzaW9uIjoiMSIsImVuY29kaW5nIjoiYnN0cmluZyIsImNvbXByZXNzZWQiOnRydWUsImVuY29kZWQiOiJ4nO1d2VbjSrJ9r69g0Vx1MDAwZv1yrM6MnM+bmYqhXHUwMDAwXHUwMDE3Zu7biyUsYVx1MDAwNMYysszUXZ/S33D/6X5JR5rCkmzJslx1MDAxOcquvvisU1VYUmYqM/aOISOSf35ZWFiMXHUwMDFmO/7in1x1MDAwYov+Q8NtXHUwMDA1XuTeL/5hv7/zo25cdTAwMTC28Vx1MDAxMvR/7oa9qNG/8zKOO90///Y3t9NxkqecRnjz/KTf8m/8dtzFe/+OPy8s/LP/Z6qvyG/EbrvZ8vtcdTAwMGb0LyXdUcKGv91cdNv9rqkmklx1MDAxOcFcdTAwMTRcdTAwMWTcXHUwMDExdFewv9j38PKF2+r6yVx1MDAxNfvV4p1+PPNXq6T6dFx1MDAxOFx1MDAxZNTIelM9NUnS7UXQatXjx9bzi7mNy16UXHUwMDFhVDeOwmv/KPDiS9v70PeD57ohTkHyVFx1MDAxNPaal22/2808XHUwMDEzdtxGXHUwMDEwP/ZfkFxmvn2ehT9cdTAwMTeSb1x1MDAxZfAnLpOfXHUwMDFmn997qO/lsFx1MDAxNUa2779cXFx1MDAxMN0gqTc6d1x1MDAxYtdNXHUwMDFjQttcdTAwMWLcXHUwMDEzR26723EjXFyU5L77n2/FUqO59IPmZWyXPDWArt+fXFwqhTFAXHJcdTAwMTeDK7abzobXX+h/JFNcdTAwMWG5N/6GfaTda7XSs9L2fs5K5sK5vbA6LDNpucnITuw/JG+RWujN3Z4+ZnRFn6xdtPaku3lzb+Ti4L5cdTAwMWY//5WMs9fx3Ge5oYqAZJRQrlxyXGaut4L29fBYW2HjOlx1MDAxMbX+tz/+yJfxzDgz4lxyheJccsZcdTAwMTBCjVx1MDAxMVx1MDAxM4t3/lvPuXhcdTAwMGItXHUwMDFkLVx0XHUwMDAxxo1cdTAwMTZMwJCwi1x1MDAwZlx1MDAxM3YuXHUwMDFkUDi/KPRgwEiTI/piRPKN1EZxSdVHS/57XG5nMqqwXHUwMDFk14Mnu1xmQDLfrrk3Qesxs4x9qcVZ3Fx0vZSg2G+rraBp5XexgaP1o4xox1x1MDAwMWqBwVxycdhJrjawXHUwMDFmN2j70cYkdFx1MDAxY0ZBM2i7rf3cYeBL++sv60Sd1Eqdu13fXu2Lz1hY9u/KhyXVhcCUlGnOeUpeyoDZqNbdxsUtQqd5+lx1MDAxMO5HvWC9XHUwMDFlzj0wXHQ4RnJGtDKMaSkzwEQhdIjiXHUwMDE0QCqDXFzFinFK+p+xOP2Lb/yGfzGKUVx1MDAwMO4oIVx0cFwiQUhJ+ChIKdWOZMC5lkxyTSGB5k/QSiZcdTAwMTgz1CS2RFx1MDAxYbNcdTAwMDO1sFx1MDAxOGntevFpJapcdTAwMDQ9XHUwMDEzP1x1MDAxZT6uV89XU/plsVx1MDAxN+yHLb4k71ZWK6d8JWqzlVx1MDAxYpG+YWOptnRcdTAwMTefPZ7s3HI3+LZz9Vx1MDAxNNVcdTAwMGVfXHUwMDE0z1TkkOi7XHUwMDE3cYWf3/yYmjNcYlNgOIBSafhMylx1MDAxOd3YjeKloO1cdTAwMDXt5vAjftsruNJyu/FyeHNcdTAwMTPEOIxaXHUwMDE4tOPhO/rtVqMovL/03ZG5wJZcdTAwMGKvdWxz2bVL/rWQyHH/h8G///FH7t1cdTAwMTXqoOahWqI9I6g0QkP6ec5cdTAwMWRcdTAwMDFaKqpQtFx1MDAwNKFlzVx1MDAxMUcxKVx1MDAxMDNcdTAwMTJcdTAwMDCYXHUwMDEwSqZb08aRqDsklVx1MDAxYaQ0hpU3R1x1MDAxOOeKYItcXHJcdTAwMWMhZEZnXGYqT8MkQ6UkKJDS9ih1XHUwMDA04dyA0iCUyTRGXHTva0RcdTAwMGXCILahtDGBU8c1KFCowGVKVz+3plx1MDAxZIJgxEFRalx1MDAwMajLx6Yt41xiXHUwMDA0vOFcdTAwMDZZhWXbM1x1MDAwZVWCXGIqXHUwMDE4amEjZWl7mjlcXCHsNadaM6V4tj3lYEPWilXC9lfaXHUwMDFlUOUoLnBcdTAwMTDUUIZcbkBm21x1MDAwM0drdFhcYjJcIr40L31fXHUwMDE0XHUwMDEwRyiNUsfRk9FCiqGlpejfXHUwMDE4YiROXHUwMDFlXHUwMDE1UN5cdTAwMWNDNajwRTWyNjc001x1MDAxYXHwVVx1MDAwNXAtUDLLxdiOTVx1MDAwMc5cbkHBwuXNvqtCTFx1MDAwMFx1MDAwNVxcVqI05YqJ8vZQ8Fx1MDAwMIfGXGLONEiRWYtcbnFcdTAwMTguO05cdTAwMDHHJTFcZpejtEFcXD7rXHUwMDAyoobAtSBcImH9foOoNlx1MDAxMGREKHRVXHUwMDE4XGJeOntcYkrHKGlcdTAwMTBOiDOUmExzVlTQJqCU4Fx1MDAxZDjD5c2h9OOyXHUwMDFhXHUwMDBic9tcXGZpK8ZBKWdGIeJcdTAwMDSlsrQ1YVUx0jfOkVJW/DKtYU+4XG7GoF2KoFx1MDAxNpyXNWdcdTAwMWNUqUrh9FGJqNCZta0wRzODy8BcZiAypOKlPIDLh9yjXHUwMDE0cijONZKUzLJcdTAwMWVeJ4Ihg1x0+76EcCVLwVFE4l/Sf1x1MDAxN1x1MDAxOXZ+q1x1MDAxNXS6XHUwMDA1tp1cdTAwMTaFTlx1MDAxN4JGKtSWk5t2q1x1MDAxYpw1xUr49Ss5k7x3qO98cT7/plx1MDAxZC5I+jNk2+Flylx1MDAxOFx1MDAwMkhcYqCUv82001L7aTc0bdohIVx1MDAwMPJcdTAwMDd2iF5NjvuFhMaVlXuOXHUwMDEwN1xi9lHLTlnX8bey7F5cdTAwMTPweFx1MDAxNrbQnMGurp+c9+r11uZ6VPvm02/Z+MdcdTAwMGJcdTAwMDJcXGs7TVx1MDAxNfFAW1x1MDAxZad4XG5cdTAwMDNxLFx1MDAwMlx1MDAwYl2rtN80hD5NUDsgJ06OvtODZeNcdTAwMWR8jTb3eD00h16j5Vx1MDAxZDfmXHUwMDFlfSpcdTAwMGI3XHUwMDAwhJvs61x1MDAwMoFcbk2YN8Gt4Xvcc0fhpp5VrI1cXFj7lIhRuDFhXHUwMDE1iVLo8qF5akhO9MPaosyq1VxcvP2i6Fx1MDAwN1x1MDAwZYLKKVx1MDAwNPX/iyeD5qnQaO1KS5dI20P289DVUlx1MDAxM5A5SP+KMqm0Jlx1MDAwNF1cdTAwMWa/QjNmzKi8TGDHoOqnaIIydOmZ4OPbK2uuUKbz3rfc/1x1MDAxOFx1MDAxZZ2cavbfZLBcdTAwMTTSpYTiXGIxOkjWT5NcdFx1MDAxOMv48rixXW/X2EZTuHV9XHUwMDE4alx1MDAxYa7defPOl9K6mehcbnFlXHUwMDE0MkBcdTAwMGV7MvSkXHRcdTAwMWFcdTAwMDKGoDPFhlx1MDAwNpZQI/Xtf+9CnznGXG5KL9oyaDiji0DRR03R+Evo2DqNgHb6XGbJk1lcdTAwMDWcXGbgkzxfPlSij8JcdTAwMTU30lx1MDAxMOtywzCZMGlcdTAwMWRcIoFcdTAwMDYrJ+X+blx1MDAwNd1dXFxtJrV13m1QRafbY9wx9lx1MDAwMrXen0BcdTAwMGasNPggtaNcdTAwMDFJ0TBcdTAwMWJbIHRcXGulo8PGhEY1XHUwMDBlXHUwMDA0m1RoiVx1MDAwZjvjoKXdeoLnm0qd8Vx1MDAwZuK+hIFHuY8oxpFcdTAwMTIm576loHvbuNvpXexvsd3KfmVlj1x1MDAxZap55z7rqSHvcUUox5WSSSsv3IdcdTAwMGW/XHKcoOiCZmpoYCnuM4o2YJamI76KXHUwMDAyXHUwMDBlfLbsh5o9mcNP9nv5fNp6v87WK0pcdTAwMDZgpNAz5lx1MDAxMvWI0WZyz/jhvmKOO+p2ldT29jerlfrjXvVk7tmOaVx1MDAwN0BriU4mqttUXHUwMDE4rlx1MDAxZpZcdTAwMTLoJ2MjoFx1MDAwMSXUqCSE/Fx1MDAxYUtvbGpcdTAwMDBcdTAwMTXSQY/cXHUwMDEwYaP9StPEXGJcdTAwMWZQXHUwMDFlXHUwMDFmzYrRXHUwMDAyXHUwMDE52cJoRlx1MDAxNDfY50s2r1+XXHUwMDFikHKvS3JcdTAwMDNqbnTb8+OFMFpY89340o/+p12Pw8htfkTKQOY1hvNcdTAwMDMmXHUwMDFhykRpXHUwMDAzjL9cbsBcIlx1MDAxNcNcdTAwMWPJ5lF2+0XpyZPVNnuSNq9cdTAwMWa19/30sLVy2ex0N6PDd0awXHUwMDE3WsX0nlx1MDAxMDZoWlx1MDAxYcaE4c/BqqzBwvtcdTAwMWJcdTAwMTmS8TGGyluhXHUwMDBi4FC0XynkmCnUOCMhZG437NCUnKVdwrnd0/wg0CZs9Fx1MDAwMtr/+/f/LsR++ymIKu2Pye1cdTAwMTlcdTAwMGLUwu6HwZmHTSpehU2qXG6VK1UgjeZ0ikTStWu1XG5dr+cu7d5eXHUwMDFkeDyq79aKtOuQlpxdXHUwMDFhKXdcdTAwMDBcdTAwMDHZd5tcdTAwMThcZnlcdTAwMTKoV1x1MDAxZLvRKyi3YWhRvOUzSVx1MDAxMOXCUz7lOVs+XHUwMDA2vWLTT/NDXHKbXHUwMDA3UJ5cdTAwMTNyplx1MDAxMlxyP51cdTAwMWHwr8cnMMKSaOor8ckmVqrbblx1MDAxYpWWt1BcdTAwMGI6ftY7/jVcdTAwMDAt7n8y9Vx0r4IomOJcXG9cdTAwMDY2XHUwMDBigU6OUL/ndZtcdTAwMWS4Pjw71l+b11wiprf1s/lGqFx1MDAwMWVcdTAwMDMvRNj0IaFcdTAwMTNcdTAwMWLuXHUwMDE3XHUwMDAylFxih3GmtWKIVUJTXHUwMDAxmHFcdTAwMTBF7kTf3pBZQpSj3atftSv0Koguf9uYXHUwMDE5PPP7flx1MDAwZmg+71x1MDAxYedhk1x1MDAxNm9DXHUwMDE4wmzSXHUwMDBln1x1MDAxY5zj97HnXHUwMDEynMImojKbtcMlQlRnXVNcdTAwMTDa0VxuNVx1MDAxYkFVgcavflsgrlxinjnFXHUwMDE5zDhcdTAwMDS1XHUwMDEzWr3AXHJcdTAwMDI3ydN6QadcdTAwMDGmmVx1MDAxNrNcdTAwMDUnMZK/KmF9KO6Wyoq9XGJcdTAwMWI92y9xmEbXSiBd4fRcdTAwMWKZzntbbLqdPkE5RKBzzG1cdTAwMWUgpamiooVBhdJzSvrhZq/ldq9P60dxle1cdTAwMDTL+kb39l/SKJLE20zEL2dMXHUwMDE14iiFy2HAhlx1MDAwNFx1MDAwMEdcdTAwMDUjg5KOQOhcdTAwMTDOJWWGcmZE0aDyc4xGXHUwMDA29dtcdTAwMDRcdTAwMWIzd49cbnDy+Jf030WcNb5+jFx1MDAxN7vkiGPOcI1cdTAwMTKro4y3rqq3sq53b0xna6lcdTAwMTPI8OGBbOt5XHUwMDBmqulUvdjc1I9cdTAwMTnAlUHYfnikbCBMU9WPrZ9W1m6uTk6+s+/u+vVWuFx1MDAxZPHq7XTZVFxupfnj68dYsVpmyLlaism1cv5Lz7l041x1MDAxZL9X/Vx1MDAxOEFDichZh5vewVaeff1YXHRcdTAwMWTPtH6MXHUwMDE314/ZQiRcdTAwMWLUmFx1MDAxOJjxwc3WgTy9+Xp2d3FcdTAwMTfV99qizo7mXHUwMDFkmFx1MDAwNv5rysfA1nRcdTAwMTCW9nDys4z3wl1/+by2XHUwMDE38NVWbfvgkF/T6no6iVx1MDAxOFZ7N6SmjtdqlcZJ3FW3XHJ2fJO+ofV0uL1d6Z0qXG7+49bB0+6l/r478/qxt3HGf/nW+Wf92ODaZ/3Y2PY+68dePp/1Y6XNfdaPZa7PU/2YLnS6bHRcdNhcdTAwMTRHdlx1MDAwNFvaa38nze/8iD+pzreGt1S4kThHpl02ovA71Ytp5GvGXHUwMDBi4lx1MDAwZfNpyb2+Xux+Za1cdTAwMGXx2tZVbVlt+Ncnx6dcdTAwMTdbK+9WL2ZccsJpXHUwMDEyjcZcIq4wKY5cdTAwMTXv20vGkaykntyTalcr20FcdTAwMGK2d1x1MDAwZjuPXHUwMDBm4cnpyu16Z+6rNdGinCYrrlx1MDAxMHwzSorDXHUwMDA1Qj5cdTAwMDf521x1MDAwNzw+k+LsV9NcdTAwMDKYp2pKR5Li0C/XWk1Rbp2/PzPnXHUwMDAwRsfKIdpmhlx1MDAxMjTmdFx1MDAxMoPob1x1MDAxZDKC9lx1MDAxMto2aJhcdTAwMDF6JzCmfmmCrcOiSFxiR/tdKHSlctPimIOGukZ/jlx1MDAwMbGp2iObiDZSQ5RkXHUwMDFmXHUwMDBl4nHqbL7Kn3MpgjpcdTAwMTI9LuvTSmzWiIlT8vyHTlx1MDAxOMVcdTAwMDXs0PIv4lx1MDAwZuCG4T4nSr+jY1mgMFwiylIlwyM7XHUwMDE1nFx1MDAxM3R4xeS1PFx1MDAxN/vR+XG0+63VPupcbk/64cn9XWveaYBcIts56LRcdKtcdTAwMTBcdHJBgsSXYlx1MDAxZUSYXHUwMDA2Llx1MDAwML0qXCI/ppBR5+zQ5aWycyPQu5ut1sa5gM8jq0bvhsztMN2GOy+9XU55v1x1MDAxZVx1MDAxYU72bJHSUFG2u9xKpKle8N3qXHUwMDBmmSxMXHUwMDFlQF2BMEU/cnLf42nrqiq7XHUwMDFkX157WndPz8DbqV7PPWcx+H04i1x1MDAwYmuiXHUwMDEwNcvDKT4565OzZslZXCK1kzpcdTAwMWEvYVx1MDAwNpTik7tbN1x1MDAwN6xcdTAwMWRvb3tGhGbrYoNcdTAwMWPodnVv7jlcdTAwMGKtSYdyQEpiXHUwMDAybfGhlFx1MDAxMMtZXHUwMDFjWcJuXHUwMDEwgU5cdTAwMTfrvS9lZSUq6SWduqm43XqT6Nhpki5n/EloSlx1MDAwYvRcdTAwMWJcdTAwMDWbVT3hJ6HZT+HuoK29J1x1MDAxY4BcdTAwMTK7mZXZU6HDa1vWWoXiXHUwMDAzaHMrpVxy/q/QvsgwzLTtqezoXHUwMDE4jGutvPhaOVpzdFx1MDAwN2xcIoXdXHUwMDA3zLyro23yrdI2t0lRXb5R+yHl06L4yGboo0hoOrm1XHUwMDE27InLK/fobMlbutmpVqu1amM3fmfme//qS1xuU/yygLfGg1FcZlAqpqi0pMqeNmL0x//WgHF8Zuw5Lq/hszeWWi789TnesvCvXHUwMDA1x3H+Oruiy8KBvEP5ZbFdXCJcbvdxgFx1MDAxMI1klN7pKUOn+1x1MDAwNOzMO7rd3VxcVlW2/3BQe3zanXu7XHUwMDA0zVx1MDAwMMcwXHUwMDBlVHElQZnRg6ykXHUwMDA2Qqk9x4JcdTAwMTI15niDN1x1MDAxY+YymS/FlCBGK5gpUj8zzj5do/ezXHUwMDBmpCm0XHUwMDBmNFx1MDAxYS3cXHUwMDFloTQxXHUwMDAxXHUwMDFkgVxcOj+/ulx1MDAxNmtcdTAwMWU5q5w/VdjX3txcdTAwMWY8ikZcdTAwMWE4jCnCbVx1MDAwZVx1MDAxNWcps/55J4rZk1x1MDAxM6hcdTAwMTFGXHUwMDAw/kHfRkCFO1HSXHUwMDExXHUwMDFhinaihGPQWDOC2zP0TDqpPonySEnox5ezXHLEbu5cdTAwMTMrci1cdTAwMTSbyydtajUx0E/tmtRi+W12othYXHUwMDFhKCxlpTjNhaZcYmXIUzB5zYz4dtlYOriq3K9CuCc3N+9Pg2Mz/zygx1x1MDAwNnWteyi54ppRQVx1MDAxNMihgaUgTuyRaq+jXHUwMDAxXFxcdTAwMDXsxSZcdTAwMGYqolx1MDAxOOM8p9R8NMRrc6JccnmPsrF3z4/XXHUwMDAyX+TTWlx1MDAxObm7eKGHXHUwMDFlf5N217p4r4baX5fFiZ48LZN+i/mV12Zfo42v0ffVx50t/2juf1EgpUaiXHUwMDAyt2Xo0D/INItqVK1SKE1AcMLYmFxu9Vx0UD2+XGKU2eiSsXmY2lx1MDAxZVaal6iZg2ybMUGFmm2sXHUwMDEzXHUwMDA38dZjXiZPXHUwMDEz21+u/eooQKbLyUrgxqd9XHUwMDE1n1x1MDAxOKGLzW3GXHJhU1jb482tuTwvglx1MDAxMjCO6X+kpoykYviZXHUwMDAzI95kYr/rOVx1MDAxMVTYeLAycqZcYny17Vt8UETmLIXxjttC+lhcdOBUclwiNVNcdTAwMTR5Qcj0oVx1MDAxMc9HOFx1MDAxOFvpwLVcdTAwMTZCoHerX2pOys+JyFxmaXxcckBqSLZcZoLapeNAUDpcdTAwMTRaq0SPXGZcblx1MDAxY2D2jGpcdTAwMWOXYVx1MDAwMlxiVyPD+m1sh1eeXHUwMDE08eVng4tup1OPUcJcdTAwMDazv3hcdTAwMTf490t5UOp/rJ/WnybLKX7fTPzx5cd/XHUwMDAwKCNcdTAwMWW8In0=Node Parquet or Feather Storage ❯ tenzir-node Managed Pipeline CLI Pipeline Node Parquet or Feather Storage export ❯ tenzir 'export | ...' export TCP
Let's bring back the historical data we imported in the previous
section :
Think of export
being the entire data at a node. As this can grow quickly, you
may query only subsets of it, e.g., by filtering it using
where
:
export | where orig_bytes < 1 KiB
Logically, this query would first export the entire historical data, and
then begin filtering the data. But since Tenzir does predicate pushdown , the
pipeline executor will analyze the query and push the
expression in where
with the predicate
orig_bytes < 1 KiB
"down" to the export
operator. Tenzir's storage engine
then asks its catalog to identify the relevant subset of partitions that the
query should execute on. This dramatically improves the query performance for
selective workloads, such as point queries for single values or specific time
ranges.
To figure out the shape of the data to query, you can show available
schemas .