Apache Parquet’s performance does not come from magic - it comes from a carefully designed physical layout that enables query engines to read the minimum amount of data necessary.
Visión general del formato y de la lectura eficiente
The file starts and ends with the magic bytes PAR1. The actual data lives in row groups in the middle. A reader always starts by reading the last 8 bytes of the file to find the footer length, then reads the footer to learn the schema, row group locations, and column statistics. A row group is a horizontal partition of the data. Predicate pushdown: Each row group carries column statistics (min, max, null count).
You do not have to take row groups on faith. In Parquet Explorer, the metadata inspector shows you the row group breakdown for any file you load: how many row groups exist, how many rows each contains, and the compression codec and column-level statistics (min/max values, null counts, distinct counts) for each column chunk. For most workloads, the default is fine.

Organización interna: columnas, páginas y compresión
Within each row group, data is stored by column. Each column chunk is stored contiguously on disk. Each column chunk has its own compression codec. Column chunks are further divided into pages, the smallest unit of I/O in Parquet. Pages are typically around 1 MB. Data pages contain the actual column values, encoded and optionally compressed.
Repetition and definition levels: Used for nested data (lists, maps, structs). There are two versions: Data Page V1 and Data Page V2. When dictionary encoding is used (the default for most writers), the first page in a column chunk is a dictionary page. Parquet supports optional column index and offset index pages that enable fine-grained filtering within a column chunk.

Codificación de valores y diccionario
Parquet uses several encoding schemes to represent column values efficiently. Values are stored as-is in their native format. The most common encoding. A dictionary of unique values is built, and each value is replaced with an integer index. Dictionary encoding is extremely effective for columns with low to moderate cardinality - status codes, country names, categories, boolean flags. Consecutive repeated values are stored as a (value, count) pair. Effective for sorted data or columns with many repeated values. Stores the difference between consecutive values rather than the values themselves. Highly effective for sorted integer columns (timestamps, auto-incrementing IDs).
A newer encoding for floating-point data. CodecSpeedRatioNotesSnappyVery fastModerateDefault in many tools. Good for interactive workloads. ZstdFastHighBest overall balance. Increasingly the recommended default. GzipSlowHighLegacy choice. Zstd is better in almost all cases.LZ4Very fastLowerFast but lower ratio than Snappy in practice.BrotliSlowVery highRarely used for Parquet. Compression is applied per page, after encoding.

El footer y el filtrado primero
The footer is the most important part of a Parquet file for query planning. Statistics: Min value, max value, null count, distinct count (optional). When a query engine opens a Parquet file, it reads the footer first. Understanding the theory is useful, but seeing it in your own files makes it concrete. All of this runs in your browser with zero installation. Sort your data before writing. Sorted columns have tighter min/max ranges per row group, which improves predicate pushdown.

Diseño físico y rendimiento de lectura
Choose column order thoughtfully. Monitor dictionary encoding fallback. If a column falls back from dictionary to plain encoding, its compression ratio drops. Right-size your row groups. The default is usually fine, but if you are optimizing for very selective queries (reading 0.1% of rows), smaller row groups help. Use Zstd compression.
Parquet’s performance comes from its layered architecture: row groups enable parallelism and pruning, column chunks enable projection pushdown, pages enable fine-grained I/O, encodings minimize data size at the logical level, and compression reduces it further at the physical level. To explore the structure of your own Parquet files hands-on, try parquetexplorer.com. Load a file and browse the schema tree, inspect row group metadata, run the data profiler, and query the data - all in one place, right in your browser.

Parquet en ClickHouse y optimización de escritura
We believe that querying data in Apache Parquet files directly can achieve similar or better storage efficiency and query performance than most specialized file formats. In this article we explain several advanced techniques needed to query data stored in the Parquet format quickly that we implemented in the Apache Arrow Rust Parquet reader. Together these techniques make the Rust implementation one of, if not the, fastest implementation for querying Parquet files - be it on local disk or remote object storage.
Apache Parquet is an increasingly popular open format for storing analytic datasets, and has become the de-facto standard for cost-effective, DBMS-agnostic data storage. Increasingly other systems, such as DuckDB and Redshift allow querying data stored in Parquet directly, but support is still often a secondary consideration compared to their native (custom) file formats. Before diving into the details of efficiently reading from Parquet, it is important to understand the file layout.
Lectura y escritura con ClickHouse
For our examples, we continue to utilize the UK house price dataset. This contains data about prices paid for real estate property in England and Wales from 1995 to the time of writing. We distribute this in Parquet format in the public s3 bucket s3://datasets-documentation/uk-house-prices/parquet/. We read and write local and S3 hosted Parquet files using ClickHouse Local. ClickHouse Local is an easy-to-use version of ClickHouse that is ideal for developers who need to perform fast processing on local and remote files using SQL without having to install a full database server. Most importantly, ClickHouse Local and ClickHouse Server share the same code for Parquet reading and writing, so any details apply to both.

Impacto de la predicación y las filas de grupo
Understanding the Parquet file format allows users to make decisions when writing files, which will directly impact the level of compression and subsequent read performance. At its high level, a file is separated into row groups. This contains up to N rows, determined at the time of writing. Within each row group, we have a chunk per column - with each containing the data for its respective column and thus providing the column orientation. While it is theoretically possible for the number of rows to be different per column chunk, we assume this is identical for simplification. These chunks are composed of pages. It is within these data pages that the raw data is stored. The maximum size of each data page is potentially configurable but not currently exposed in ClickHouse, which uses the default value of 1MB.
We visualize estos conceptos y la estructura lógica debajo. Para ilustrar, asumimos un tamaño de grupo de filas de 6, con 11 filas en total, cada una con 3 columnas. Hay dos tipos de páginas: data pages y dictionary pages. Dictionary pages result when dictionary encoding is applied to values in a data page. For ClickHouse, this is enabled by default when Parquet files are written. For data pages that have been dictionary encoded, they will be preceded by the dictionary page. This effectively means dictionary and data pages alternate, as shown below.

Optimización práctica y recomendaciones
A limit can be imposed on the dictionary page size, which defaults to 1MB. The above is a simplification of the Parquet format. Note that while Parquet is officially described as column-based, the introduction of row groups and the sequential storage of chunks of columns means it is often described as a Hybrid-base format. In addition to storing data values, the Parquet format includes metadata. As well as storing the data schema, and information to assist with decoding, e.g., offset values and encodings used, Parquet includes information that query engines can exploit to skip column chunks. A reader is first expected to read the file metadata to find all the column chunks they are interested in before only reading those required sequentially. Furthermore, statistics can be included at the row group level describing each column's minimum and maximum value. This allows a reader to consider this information against any predicate (WHERE clause if querying in SQL), further skipping column chunks. This predicate pushdown is not currently implemented in ClickHouse but is planned for addition.
Finally, the official specification allows for separate metadata files referencing multiple Parquet files, e.g., one per column. When writing Parquet files with ClickHouse, users may wish to control the number of row groups that are written to increase the amount of parallelization in reads - see “Parallelised Reads” for more details. As of the time of writing, we recommend users use the INSERT INTO FUNCTION

Exportación, ordenación y compresión
For SELECT queries using the FORMAT clause e.g. SELECT * FROM uk_price_paid FORMAT Parquet, as well as the INTO OUTFILE clause, the factors determining the group size are more complex. Most importantly, these approaches tend to result in a very large number of row groups in files - potentially negatively impacting compression and read performance. The differences in these two approaches to exporting Parquet files can lead to varying file sizes depending on the approach and the resulting effective compression. For readers returning from our previous blog post, this addresses why different queries produced variable file sizes - smaller row group sizes don’t compress as well. In general, we recommend the INSERT INTO FUNCTION
Other tools, as well as the official Apache Arrow libraries (used by ClickHouse) for writing Parquet files, allow the number of row groups to also be configured. Parquet is a binary format where column values are stored with a specific type of either boolean, numeric(int32, int64, int96, float, double), byte array(binary), or fixed length binary. These primitive types can be annotated with information specifying how they should be interpreted to create “logical types” such as String and Enum. For example, the logical String type is encoded in a byte array with an annotation indicating UTF8 encoding.
Dictionary encoding builds a dictionary of all the distinct values of a column, replacing the original value with its respective index. This is particularly useful for categorical data, such as country names, user roles, or product categories, where repeated values are common. Imagine you have a dataset with millions of rows, where the “Country” column only contains 200 distinct values. Storing the full string repeatedly wastes space and slows down queries. With dictionary encoding, Parquet replaces each country name with a small integer reference, significantly reducing the data size. This encoding is so efficient that it is often the default for columns with low cardinality.

Conclusión operativa
Parquet’s diseño físico, desde los row groups hasta las páginas, las codificaciones y la compresión, está orientado a reducir I/O y CPU durante las lecturas de consultas analíticas. La elección de tamaño de row group, el uso de diccionario, y la selección de codificaciones adecuadas pueden impactar de forma significativa la velocidad de las consultas y la eficiencia de almacenamiento, especialmente cuando se trabajan con grandes conjuntos de datos y consultas con filtros complejos.
tags: #parquet #dictionary #page #offset