Apache Parquet
Parquet is the standard lakehouse/warehouse on-disk format (Spark, DuckDB, BigQuery export, Snowflake). Column chunk layout explains why analytics workloads beat row-oriented JSON/CSV.
What It Defines
Columnar storage file format for analytics: magic `PAR1` header/footer, row groups, column chunks, and Thrift-encoded metadata (schema, statistics, compression codec per column). Optimized for OLAP queries — read only needed columns. MIME type: `application/vnd.apache.parquet`.
Canonical (Normative)
Convenient (Practical)
Open-source foundation publishing columnar and analytics file formats widely used in data pipelines: Apache Parquet (columnar storage) and Apache Arrow (in-memory columnar + IPC serialization). Specs are maintained on parquet.apache.org and arrow.apache.org.
Related Specs
Arrow is the interchange layer between Python/R/JVM analytics, DuckDB, and Parquet readers. IPC framing is how columnar data crosses process boundaries without JSON overhead.
Protobuf is the wire format for gRPC and many high-performance internal APIs. Schemas are also used for event/message contracts in Kafka/pubsub systems.
JSON is the lingua franca of web APIs. RFC 8259 is short and worth reading once — it clarifies edge cases around numbers, encoding, and trailing commas.