3 posts tagged with "parquet"

Apache Arrow and Apache Parquet have become the de-facto columnar formats for in-memory and on-disk representations when it comes to structured data. Both are strong together, as they provide data interoperability and foster a diverse ecosystem of data tools. But how well do they actually work together from an engineering perspective?

How does Apache Parquet compare to Feather for storing structured security data? In this blog post, we answer this question.

Apache Parquet is the common denominator for structured data at rest. The data science ecosystem has long appreciated this. But infosec? Why should you care about Parquet when building a threat detection and investigation platform? In this blog post series we share our opinionated view on this question. In the next three blog posts, we

1. describe how VAST uses Parquet and its little brother Feather
2. benchmark the two formats against each other for typical workloads
3. share our experience with all the engineering gotchas we encountered along the way