Which data file format fits (CSV/JSON/XML vs Avro vs Parquet)
Verdict: Pick delimited text, JSON, or XML for human-readable exchange across many applications. Pick Avro for write-optimised row storage with a JSON schema header. Pick Parquet for read-optimised columnar analytics.
| Criterion | CSV/JSON/XML | Avro | Parquet |
|---|---|---|---|
| Structure | Human-readable text, structured or semi-structured | Optimised binary, row-based | Optimised binary, columnar |
| Optimised for | Wide, human-readable access across many applications | Compression and reduced storage/network bandwidth (writes) | Column scans and compression (analytics reads) |
| Schema | CSV: none; JSON/XML: self-describing tags/pairs | JSON header at file start describing the record structure | N/A; values for each column stored together in row groups |
Rules
- Delimited text such as CSV suits structured data that many applications must read in human-readable form.
- CSV, JSON, and XML are grouped as human-readable text formats; Parquet and Avro are grouped as optimised binary formats.
- XML represents elements and attributes with tags enclosed in angle-brackets.
- Parquet is columnar and stores each column's data together within row groups.
- Avro is row-based, and its schema header describing the data structure is stored as JSON.
Traps
- Do not swap the pair: the JSON schema header belongs to Avro (row-based), not Parquet (columnar).
- Avro compresses well for storage and network bandwidth but is binary, so it is not human-readable across a wide range of applications like CSV.
- Parquet and Avro are not uncompressed plain text like CSV; both are optimised binary formats.
- BLOB is raw binary for unstructured media, not a human-readable structured format like CSV, JSON, or XML.