Skip to content

Conversation

@lyang24
Copy link
Contributor

@lyang24 lyang24 commented Jan 23, 2026

Which issue does this PR close?

  • Closes #NNN.

Rationale for this change

parquet reading perf - if rle value is true and rle left have enough room for the current batch. lets skip the decode loop the overhead of count_set_bits for null bitmap.

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions bot added the parquet Changes to the parquet crate label Jan 23, 2026
@lyang24 lyang24 changed the title enhance: rle skip null bitmap on batch contains all max levels enhance: rle skip decode loop on batch contains all max levels Jan 23, 2026
@Dandandan
Copy link
Contributor

run benchmark arrow_reader

@alamb-ghbot
Copy link

🤖 ./gh_compare_arrow.sh gh_compare_arrow.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing rle_fast_path (f5a1966) to 3c6ca57 diff
BENCH_NAME=arrow_reader
BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench arrow_reader
BENCH_FILTER=
BENCH_BRANCH_NAME=rle_fast_path
Results will be posted here when complete

@alamb-ghbot
Copy link

🤖: Benchmark completed

Details

group                                                                                                      main                                   rle_fast_path
-----                                                                                                      ----                                   -------------
arrow_array_reader/BYTE_ARRAY/Decimal128Array/plain encoded, mandatory, no NULLs                           1.00  1189.4±19.22µs        ? ?/sec    1.07  1272.4±11.82µs        ? ?/sec
arrow_array_reader/BYTE_ARRAY/Decimal128Array/plain encoded, optional, half NULLs                          1.00  1264.6±58.91µs        ? ?/sec    1.02   1293.6±5.79µs        ? ?/sec
arrow_array_reader/BYTE_ARRAY/Decimal128Array/plain encoded, optional, no NULLs                            1.00  1200.2±52.85µs        ? ?/sec    1.06   1276.6±4.92µs        ? ?/sec
arrow_array_reader/BinaryArray/dictionary encoded, mandatory, no NULLs                                     1.00    477.5±7.57µs        ? ?/sec    1.03    489.5±6.13µs        ? ?/sec
arrow_array_reader/BinaryArray/dictionary encoded, optional, half NULLs                                    1.00    647.7±2.38µs        ? ?/sec    1.00   650.7±10.47µs        ? ?/sec
arrow_array_reader/BinaryArray/dictionary encoded, optional, no NULLs                                      1.01   487.8±12.70µs        ? ?/sec    1.00   484.3±12.29µs        ? ?/sec
arrow_array_reader/BinaryArray/plain encoded, mandatory, no NULLs                                          1.00    547.7±6.80µs        ? ?/sec    1.05    573.8±3.35µs        ? ?/sec
arrow_array_reader/BinaryArray/plain encoded, optional, half NULLs                                         1.00   718.8±12.89µs        ? ?/sec    1.02    732.4±3.79µs        ? ?/sec
arrow_array_reader/BinaryArray/plain encoded, optional, no NULLs                                           1.00   557.4±16.46µs        ? ?/sec    1.05   585.2±11.16µs        ? ?/sec
arrow_array_reader/BinaryViewArray/dictionary encoded, mandatory, no NULLs                                 1.03    156.1±3.97µs        ? ?/sec    1.00    151.9±1.79µs        ? ?/sec
arrow_array_reader/BinaryViewArray/dictionary encoded, optional, half NULLs                                1.22   216.9±12.84µs        ? ?/sec    1.00    178.4±5.42µs        ? ?/sec
arrow_array_reader/BinaryViewArray/dictionary encoded, optional, no NULLs                                  1.00    153.8±1.94µs        ? ?/sec    1.02    157.5±4.45µs        ? ?/sec
arrow_array_reader/BinaryViewArray/plain encoded, mandatory, no NULLs                                      1.03    218.9±4.12µs        ? ?/sec    1.00    211.6±6.73µs        ? ?/sec
arrow_array_reader/BinaryViewArray/plain encoded, mandatory, no NULLs, short string                        1.00    233.5±0.75µs        ? ?/sec    1.00    233.1±1.75µs        ? ?/sec
arrow_array_reader/BinaryViewArray/plain encoded, optional, half NULLs                                     1.22    258.5±3.04µs        ? ?/sec    1.00    211.6±3.33µs        ? ?/sec
arrow_array_reader/BinaryViewArray/plain encoded, optional, no NULLs                                       1.05    228.7±5.32µs        ? ?/sec    1.00    217.8±6.35µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/byte_stream_split encoded, mandatory, no NULLs     1.04  1074.8±23.56µs        ? ?/sec    1.00  1035.7±17.67µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/byte_stream_split encoded, optional, half NULLs    1.01    942.1±7.05µs        ? ?/sec    1.00   934.0±10.62µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/byte_stream_split encoded, optional, no NULLs      1.04  1083.1±10.49µs        ? ?/sec    1.00  1040.6±15.45µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/plain encoded, mandatory, no NULLs                 1.13    456.4±7.15µs        ? ?/sec    1.00    405.3±8.85µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/plain encoded, optional, half NULLs                1.04    627.9±7.90µs        ? ?/sec    1.00    601.8±6.63µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/plain encoded, optional, no NULLs                  1.12   464.7±10.40µs        ? ?/sec    1.00   413.3±10.06µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/byte_stream_split encoded, mandatory, no NULLs        1.27    194.9±2.44µs        ? ?/sec    1.00    153.1±0.66µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/byte_stream_split encoded, optional, half NULLs       1.21    360.5±4.40µs        ? ?/sec    1.00    297.3±5.40µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/byte_stream_split encoded, optional, no NULLs         1.28    200.1±2.70µs        ? ?/sec    1.00    156.8±2.90µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/plain encoded, mandatory, no NULLs                    1.57    119.1±2.29µs        ? ?/sec    1.00     76.0±2.46µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/plain encoded, optional, half NULLs                   1.24    321.0±2.88µs        ? ?/sec    1.00    259.9±6.43µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/plain encoded, optional, no NULLs                     1.54    122.9±2.71µs        ? ?/sec    1.00     80.0±3.03µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/byte_stream_split encoded, mandatory, no NULLs                    1.00    685.6±3.75µs        ? ?/sec    1.00    687.7±6.27µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/byte_stream_split encoded, optional, half NULLs                   1.00   542.3±14.95µs        ? ?/sec    1.06    577.0±4.97µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/byte_stream_split encoded, optional, no NULLs                     1.00    693.8±6.41µs        ? ?/sec    1.00    694.4±4.97µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/plain encoded, mandatory, no NULLs                                1.11     70.3±2.73µs        ? ?/sec    1.00     63.2±5.83µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/plain encoded, optional, half NULLs                               1.00    236.7±5.42µs        ? ?/sec    1.14    270.8±4.19µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/plain encoded, optional, no NULLs                                 1.17     77.5±3.08µs        ? ?/sec    1.00     66.2±4.53µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/byte_stream_split encoded, mandatory, no NULLs                     1.00     85.6±1.23µs        ? ?/sec    1.01     86.1±0.93µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/byte_stream_split encoded, optional, half NULLs                    1.09    249.3±4.20µs        ? ?/sec    1.00    229.5±3.37µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/byte_stream_split encoded, optional, no NULLs                      1.01     91.6±1.04µs        ? ?/sec    1.00     90.5±0.45µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/plain encoded, mandatory, no NULLs                                 1.00      9.3±0.17µs        ? ?/sec    1.01      9.4±0.23µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/plain encoded, optional, half NULLs                                1.15   216.9±29.87µs        ? ?/sec    1.00    188.8±0.77µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/plain encoded, optional, no NULLs                                  1.08     14.2±0.26µs        ? ?/sec    1.00     13.2±0.16µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/byte_stream_split encoded, mandatory, no NULLs                     1.00    169.9±1.33µs        ? ?/sec    1.00    170.2±0.64µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/byte_stream_split encoded, optional, half NULLs                    1.06    391.5±5.94µs        ? ?/sec    1.00    371.0±3.56µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/byte_stream_split encoded, optional, no NULLs                      1.00    174.7±0.71µs        ? ?/sec    1.00    174.4±1.79µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/plain encoded, mandatory, no NULLs                                 1.06     14.2±0.28µs        ? ?/sec    1.00     13.4±0.32µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/plain encoded, optional, half NULLs                                1.07    315.4±1.99µs        ? ?/sec    1.00    294.5±5.41µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/plain encoded, optional, no NULLs                                  1.06     19.8±0.51µs        ? ?/sec    1.00     18.7±0.54µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/byte_stream_split encoded, mandatory, no NULLs                     1.01   343.7±13.62µs        ? ?/sec    1.00    341.5±5.06µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/byte_stream_split encoded, optional, half NULLs                    1.00   357.7±10.37µs        ? ?/sec    1.10    394.3±4.83µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/byte_stream_split encoded, optional, no NULLs                      1.01    349.0±2.81µs        ? ?/sec    1.00    347.2±4.23µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/plain encoded, mandatory, no NULLs                                 1.23     28.9±1.53µs        ? ?/sec    1.00     23.6±0.42µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/plain encoded, optional, half NULLs                                1.00    201.0±4.83µs        ? ?/sec    1.19    238.5±4.32µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/plain encoded, optional, no NULLs                                  1.12     35.6±1.33µs        ? ?/sec    1.00     31.8±0.78µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed skip, mandatory, no NULLs                           1.00    108.8±1.23µs        ? ?/sec    1.00    109.2±0.97µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed skip, optional, half NULLs                          1.01    130.6±0.95µs        ? ?/sec    1.00    129.2±0.97µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed skip, optional, no NULLs                            1.01    111.8±0.64µs        ? ?/sec    1.00    111.1±1.58µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed, mandatory, no NULLs                                1.00    158.7±1.97µs        ? ?/sec    1.00    159.0±2.43µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed, optional, half NULLs                               1.01    223.3±2.56µs        ? ?/sec    1.00    221.2±4.16µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed, optional, no NULLs                                 1.00    164.0±2.52µs        ? ?/sec    1.00    163.3±3.73µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/byte_stream_split encoded, mandatory, no NULLs                    1.00     75.9±1.44µs        ? ?/sec    1.00     75.8±0.81µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/byte_stream_split encoded, optional, half NULLs                   1.02    179.3±2.97µs        ? ?/sec    1.00    176.2±0.81µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/byte_stream_split encoded, optional, no NULLs                     1.03     82.5±0.60µs        ? ?/sec    1.00     80.3±0.97µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/dictionary encoded, mandatory, no NULLs                           1.00    133.8±1.97µs        ? ?/sec    1.07    142.8±1.16µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/dictionary encoded, optional, half NULLs                          1.00    211.9±3.88µs        ? ?/sec    1.01    214.3±2.73µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/dictionary encoded, optional, no NULLs                            1.00    139.2±2.08µs        ? ?/sec    1.05    146.7±0.62µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/plain encoded, mandatory, no NULLs                                1.01     71.5±0.63µs        ? ?/sec    1.00     70.9±0.38µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/plain encoded, optional, half NULLs                               1.01    176.8±1.86µs        ? ?/sec    1.00    175.8±7.75µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/plain encoded, optional, no NULLs                                 1.02     76.6±0.34µs        ? ?/sec    1.00     75.2±0.39µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed skip, mandatory, no NULLs                           1.00    107.0±0.49µs        ? ?/sec    1.00    107.2±3.10µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed skip, optional, half NULLs                          1.01    118.9±0.65µs        ? ?/sec    1.00    117.3±0.44µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed skip, optional, no NULLs                            1.00    109.4±0.48µs        ? ?/sec    1.00    109.6±1.74µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed, mandatory, no NULLs                                1.01    160.6±1.30µs        ? ?/sec    1.00    159.5±2.16µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed, optional, half NULLs                               1.02    202.7±1.63µs        ? ?/sec    1.00    198.5±1.29µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed, optional, no NULLs                                 1.01    163.9±0.74µs        ? ?/sec    1.00    161.9±1.34µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/byte_stream_split encoded, mandatory, no NULLs                    1.00    201.0±0.65µs        ? ?/sec    1.00    200.5±1.26µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/byte_stream_split encoded, optional, half NULLs                   1.01    228.1±2.21µs        ? ?/sec    1.00    224.8±1.06µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/byte_stream_split encoded, optional, no NULLs                     1.01    207.5±1.17µs        ? ?/sec    1.00    206.2±1.79µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/dictionary encoded, mandatory, no NULLs                           1.07    149.1±0.87µs        ? ?/sec    1.00    140.0±0.77µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/dictionary encoded, optional, half NULLs                          1.03    198.3±2.90µs        ? ?/sec    1.00    192.4±2.17µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/dictionary encoded, optional, no NULLs                            1.07    154.8±1.35µs        ? ?/sec    1.00    145.2±0.91µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/plain encoded, mandatory, no NULLs                                1.00     97.4±1.75µs        ? ?/sec    1.02     99.8±1.72µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/plain encoded, optional, half NULLs                               1.03    172.6±2.14µs        ? ?/sec    1.00    167.6±3.32µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/plain encoded, optional, no NULLs                                 1.03    108.0±1.15µs        ? ?/sec    1.00    105.0±1.50µs        ? ?/sec
arrow_array_reader/Int16Array/binary packed skip, mandatory, no NULLs                                      1.00     76.5±0.84µs        ? ?/sec    1.00     76.6±0.78µs        ? ?/sec
arrow_array_reader/Int16Array/binary packed skip, optional, half NULLs                                     1.00    103.3±0.71µs        ? ?/sec    1.00    103.1±1.40µs        ? ?/sec
arrow_array_reader/Int16Array/binary packed skip, optional, no NULLs                                       1.01     79.0±0.72µs        ? ?/sec    1.00     78.5±1.20µs        ? ?/sec
arrow_array_reader/Int16Array/binary packed, mandatory, no NULLs                                           1.00    106.3±1.39µs        ? ?/sec    1.00    105.8±0.49µs        ? ?/sec
arrow_array_reader/Int16Array/binary packed, optional, half NULLs                                          1.01    176.2±2.79µs        ? ?/sec    1.00    174.7±3.74µs        ? ?/sec
arrow_array_reader/Int16Array/binary packed, optional, no NULLs                                            1.01    111.1±2.57µs        ? ?/sec    1.00    109.9±2.71µs        ? ?/sec
arrow_array_reader/Int16Array/byte_stream_split encoded, mandatory, no NULLs                               1.04     42.4±0.35µs        ? ?/sec    1.00     40.9±0.27µs        ? ?/sec
arrow_array_reader/Int16Array/byte_stream_split encoded, optional, half NULLs                              1.00    141.0±2.70µs        ? ?/sec    1.00    140.3±0.65µs        ? ?/sec
arrow_array_reader/Int16Array/byte_stream_split encoded, optional, no NULLs                                1.06     46.5±0.20µs        ? ?/sec    1.00     44.0±0.22µs        ? ?/sec
arrow_array_reader/Int16Array/dictionary encoded, mandatory, no NULLs                                      1.00    100.8±1.02µs        ? ?/sec    1.09    109.9±0.58µs        ? ?/sec
arrow_array_reader/Int16Array/dictionary encoded, optional, half NULLs                                     1.00    174.2±1.34µs        ? ?/sec    1.03    179.6±4.10µs        ? ?/sec
arrow_array_reader/Int16Array/dictionary encoded, optional, no NULLs                                       1.00    105.2±0.45µs        ? ?/sec    1.08    113.5±1.63µs        ? ?/sec
arrow_array_reader/Int16Array/plain encoded, mandatory, no NULLs                                           1.00     36.5±0.49µs        ? ?/sec    1.01     37.0±0.42µs        ? ?/sec
arrow_array_reader/Int16Array/plain encoded, optional, half NULLs                                          1.00    139.0±2.67µs        ? ?/sec    1.00    139.0±0.85µs        ? ?/sec
arrow_array_reader/Int16Array/plain encoded, optional, no NULLs                                            1.03     41.2±0.15µs        ? ?/sec    1.00     40.1±0.56µs        ? ?/sec
arrow_array_reader/Int32Array/binary packed skip, mandatory, no NULLs                                      1.00     82.3±0.76µs        ? ?/sec    1.00     82.5±1.23µs        ? ?/sec
arrow_array_reader/Int32Array/binary packed skip, optional, half NULLs                                     1.01    101.8±0.63µs        ? ?/sec    1.00    101.3±0.89µs        ? ?/sec
arrow_array_reader/Int32Array/binary packed skip, optional, no NULLs                                       1.01     85.0±1.50µs        ? ?/sec    1.00     84.4±0.44µs        ? ?/sec
arrow_array_reader/Int32Array/binary packed, mandatory, no NULLs                                           1.00    107.1±1.46µs        ? ?/sec    1.00    106.9±1.84µs        ? ?/sec
arrow_array_reader/Int32Array/binary packed, optional, half NULLs                                          1.00    166.4±5.11µs        ? ?/sec    1.00    166.0±3.64µs        ? ?/sec
arrow_array_reader/Int32Array/binary packed, optional, no NULLs                                            1.01    111.4±0.75µs        ? ?/sec    1.00    110.5±3.92µs        ? ?/sec
arrow_array_reader/Int32Array/byte_stream_split encoded, mandatory, no NULLs                               1.00     23.1±0.27µs        ? ?/sec    1.03     23.9±0.39µs        ? ?/sec
arrow_array_reader/Int32Array/byte_stream_split encoded, optional, half NULLs                              1.00    121.2±1.54µs        ? ?/sec    1.00    121.4±2.33µs        ? ?/sec
arrow_array_reader/Int32Array/byte_stream_split encoded, optional, no NULLs                                1.02     27.4±0.35µs        ? ?/sec    1.00     26.8±0.68µs        ? ?/sec
arrow_array_reader/Int32Array/dictionary encoded, mandatory, no NULLs                                      1.00     82.3±1.71µs        ? ?/sec    1.10     90.8±0.48µs        ? ?/sec
arrow_array_reader/Int32Array/dictionary encoded, optional, half NULLs                                     1.00    154.8±0.65µs        ? ?/sec    1.03    160.2±3.20µs        ? ?/sec
arrow_array_reader/Int32Array/dictionary encoded, optional, no NULLs                                       1.00     85.6±0.47µs        ? ?/sec    1.11     95.1±2.82µs        ? ?/sec
arrow_array_reader/Int32Array/plain encoded, mandatory, no NULLs                                           1.00     15.2±0.43µs        ? ?/sec    1.04     15.8±0.91µs        ? ?/sec
arrow_array_reader/Int32Array/plain encoded, optional, half NULLs                                          1.00    119.3±0.97µs        ? ?/sec    1.00    119.7±3.16µs        ? ?/sec
arrow_array_reader/Int32Array/plain encoded, optional, no NULLs                                            1.04     21.1±0.34µs        ? ?/sec    1.00     20.2±0.45µs        ? ?/sec
arrow_array_reader/Int64Array/binary packed skip, mandatory, no NULLs                                      1.00     79.4±1.09µs        ? ?/sec    1.03     81.4±0.55µs        ? ?/sec
arrow_array_reader/Int64Array/binary packed skip, optional, half NULLs                                     1.00     89.8±1.15µs        ? ?/sec    1.01     90.4±1.04µs        ? ?/sec
arrow_array_reader/Int64Array/binary packed skip, optional, no NULLs                                       1.00     82.0±2.05µs        ? ?/sec    1.00     81.9±0.54µs        ? ?/sec
arrow_array_reader/Int64Array/binary packed, mandatory, no NULLs                                           1.00    106.9±1.30µs        ? ?/sec    1.01    108.5±1.60µs        ? ?/sec
arrow_array_reader/Int64Array/binary packed, optional, half NULLs                                          1.00    144.9±3.80µs        ? ?/sec    1.05    152.1±1.03µs        ? ?/sec
arrow_array_reader/Int64Array/binary packed, optional, no NULLs                                            1.00    110.3±4.16µs        ? ?/sec    1.01    111.2±0.69µs        ? ?/sec
arrow_array_reader/Int64Array/byte_stream_split encoded, mandatory, no NULLs                               1.00    146.0±0.72µs        ? ?/sec    1.01    147.0±0.70µs        ? ?/sec
arrow_array_reader/Int64Array/byte_stream_split encoded, optional, half NULLs                              1.00    167.3±2.28µs        ? ?/sec    1.01    169.3±2.59µs        ? ?/sec
arrow_array_reader/Int64Array/byte_stream_split encoded, optional, no NULLs                                1.00    150.7±0.57µs        ? ?/sec    1.00    150.9±0.61µs        ? ?/sec
arrow_array_reader/Int64Array/dictionary encoded, mandatory, no NULLs                                      1.10     96.5±1.04µs        ? ?/sec    1.00     88.0±1.15µs        ? ?/sec
arrow_array_reader/Int64Array/dictionary encoded, optional, half NULLs                                     1.03    139.6±1.41µs        ? ?/sec    1.00    135.4±0.98µs        ? ?/sec
arrow_array_reader/Int64Array/dictionary encoded, optional, no NULLs                                       1.11    101.2±2.71µs        ? ?/sec    1.00     91.4±1.00µs        ? ?/sec
arrow_array_reader/Int64Array/plain encoded, mandatory, no NULLs                                           1.00     37.3±0.77µs        ? ?/sec    1.02     38.2±1.33µs        ? ?/sec
arrow_array_reader/Int64Array/plain encoded, optional, half NULLs                                          1.00    110.1±0.57µs        ? ?/sec    1.01    111.6±0.77µs        ? ?/sec
arrow_array_reader/Int64Array/plain encoded, optional, no NULLs                                            1.00     42.0±0.71µs        ? ?/sec    1.13     47.5±3.22µs        ? ?/sec
arrow_array_reader/Int8Array/binary packed skip, mandatory, no NULLs                                       1.00     80.4±0.54µs        ? ?/sec    1.00     80.3±1.18µs        ? ?/sec
arrow_array_reader/Int8Array/binary packed skip, optional, half NULLs                                      1.03   105.9±11.99µs        ? ?/sec    1.00    102.9±1.45µs        ? ?/sec
arrow_array_reader/Int8Array/binary packed skip, optional, no NULLs                                        1.01     82.9±1.62µs        ? ?/sec    1.00     82.1±0.45µs        ? ?/sec
arrow_array_reader/Int8Array/binary packed, mandatory, no NULLs                                            1.00    107.7±1.08µs        ? ?/sec    1.03    110.6±1.38µs        ? ?/sec
arrow_array_reader/Int8Array/binary packed, optional, half NULLs                                           1.01    172.0±5.41µs        ? ?/sec    1.00    170.8±1.09µs        ? ?/sec
arrow_array_reader/Int8Array/binary packed, optional, no NULLs                                             1.01    112.2±0.91µs        ? ?/sec    1.00    111.2±1.72µs        ? ?/sec
arrow_array_reader/Int8Array/byte_stream_split encoded, mandatory, no NULLs                                1.00     34.5±0.32µs        ? ?/sec    1.00     34.6±0.42µs        ? ?/sec
arrow_array_reader/Int8Array/byte_stream_split encoded, optional, half NULLs                               1.00    133.0±1.44µs        ? ?/sec    1.00    132.9±1.58µs        ? ?/sec
arrow_array_reader/Int8Array/byte_stream_split encoded, optional, no NULLs                                 1.04     39.1±0.88µs        ? ?/sec    1.00     37.6±0.31µs        ? ?/sec
arrow_array_reader/Int8Array/dictionary encoded, mandatory, no NULLs                                       1.00     92.5±0.77µs        ? ?/sec    1.10    102.2±0.91µs        ? ?/sec
arrow_array_reader/Int8Array/dictionary encoded, optional, half NULLs                                      1.00    165.7±0.47µs        ? ?/sec    1.04    172.4±4.80µs        ? ?/sec
arrow_array_reader/Int8Array/dictionary encoded, optional, no NULLs                                        1.00     97.1±0.62µs        ? ?/sec    1.09    105.5±1.33µs        ? ?/sec
arrow_array_reader/Int8Array/plain encoded, mandatory, no NULLs                                            1.00     28.7±0.27µs        ? ?/sec    1.01     29.0±0.37µs        ? ?/sec
arrow_array_reader/Int8Array/plain encoded, optional, half NULLs                                           1.00    130.3±0.81µs        ? ?/sec    1.01    132.2±5.40µs        ? ?/sec
arrow_array_reader/Int8Array/plain encoded, optional, no NULLs                                             1.04     33.4±0.30µs        ? ?/sec    1.00     32.2±0.26µs        ? ?/sec
arrow_array_reader/ListArray/plain encoded optional strings half NULLs                                     1.00      6.1±0.17ms        ? ?/sec    1.02      6.2±0.08ms        ? ?/sec
arrow_array_reader/ListArray/plain encoded optional strings no NULLs                                       1.00     12.0±0.18ms        ? ?/sec    1.01     12.1±0.20ms        ? ?/sec
arrow_array_reader/StringArray/dictionary encoded, mandatory, no NULLs                                     1.00    488.1±3.52µs        ? ?/sec    1.00    485.7±4.97µs        ? ?/sec
arrow_array_reader/StringArray/dictionary encoded, optional, half NULLs                                    1.00    647.2±4.22µs        ? ?/sec    1.03   669.5±35.80µs        ? ?/sec
arrow_array_reader/StringArray/dictionary encoded, optional, no NULLs                                      1.00    480.2±5.81µs        ? ?/sec    1.02   491.4±14.97µs        ? ?/sec
arrow_array_reader/StringArray/plain encoded, mandatory, no NULLs                                          1.00    655.6±4.91µs        ? ?/sec    1.11   724.8±10.14µs        ? ?/sec
arrow_array_reader/StringArray/plain encoded, optional, half NULLs                                         1.00    763.5±4.05µs        ? ?/sec    1.06    811.1±6.10µs        ? ?/sec
arrow_array_reader/StringArray/plain encoded, optional, no NULLs                                           1.00    659.3±9.00µs        ? ?/sec    1.11    729.5±3.34µs        ? ?/sec
arrow_array_reader/StringDictionary/dictionary encoded, mandatory, no NULLs                                1.01    332.1±6.73µs        ? ?/sec    1.00    328.4±5.07µs        ? ?/sec
arrow_array_reader/StringDictionary/dictionary encoded, optional, half NULLs                               1.00    385.0±6.14µs        ? ?/sec    1.08    416.0±5.68µs        ? ?/sec
arrow_array_reader/StringDictionary/dictionary encoded, optional, no NULLs                                 1.01    336.8±6.09µs        ? ?/sec    1.00    333.8±5.92µs        ? ?/sec
arrow_array_reader/StringViewArray/dictionary encoded, mandatory, no NULLs                                 1.00    141.4±1.90µs        ? ?/sec    1.07    150.9±2.91µs        ? ?/sec
arrow_array_reader/StringViewArray/dictionary encoded, optional, half NULLs                                1.27    200.9±2.54µs        ? ?/sec    1.00    158.7±0.87µs        ? ?/sec
arrow_array_reader/StringViewArray/dictionary encoded, optional, no NULLs                                  1.03    139.1±4.29µs        ? ?/sec    1.00    135.5±2.48µs        ? ?/sec
arrow_array_reader/StringViewArray/plain encoded, mandatory, no NULLs                                      1.00    361.1±5.69µs        ? ?/sec    1.00    361.5±6.11µs        ? ?/sec
arrow_array_reader/StringViewArray/plain encoded, optional, half NULLs                                     1.12    322.3±6.71µs        ? ?/sec    1.00    287.4±2.31µs        ? ?/sec
arrow_array_reader/StringViewArray/plain encoded, optional, no NULLs                                       1.00    371.9±2.92µs        ? ?/sec    1.01    373.9±9.32µs        ? ?/sec
arrow_array_reader/UInt16Array/binary packed skip, mandatory, no NULLs                                     1.01     93.1±0.81µs        ? ?/sec    1.00     91.9±1.11µs        ? ?/sec
arrow_array_reader/UInt16Array/binary packed skip, optional, half NULLs                                    1.00    111.9±0.78µs        ? ?/sec    1.00    111.8±1.66µs        ? ?/sec
arrow_array_reader/UInt16Array/binary packed skip, optional, no NULLs                                      1.01     94.3±1.58µs        ? ?/sec    1.00     93.7±0.80µs        ? ?/sec
arrow_array_reader/UInt16Array/binary packed, mandatory, no NULLs                                          1.00    125.4±0.71µs        ? ?/sec    1.01    126.3±6.51µs        ? ?/sec
arrow_array_reader/UInt16Array/binary packed, optional, half NULLs                                         1.05   195.6±35.40µs        ? ?/sec    1.00    187.0±1.01µs        ? ?/sec
arrow_array_reader/UInt16Array/binary packed, optional, no NULLs                                           1.01    129.9±1.03µs        ? ?/sec    1.00    129.0±0.76µs        ? ?/sec
arrow_array_reader/UInt16Array/byte_stream_split encoded, mandatory, no NULLs                              1.00     40.8±0.19µs        ? ?/sec    1.05     42.8±0.61µs        ? ?/sec
arrow_array_reader/UInt16Array/byte_stream_split encoded, optional, half NULLs                             1.00    141.7±6.68µs        ? ?/sec    1.01    143.1±6.14µs        ? ?/sec
arrow_array_reader/UInt16Array/byte_stream_split encoded, optional, no NULLs                               1.00     45.3±0.57µs        ? ?/sec    1.01     45.7±0.21µs        ? ?/sec
arrow_array_reader/UInt16Array/dictionary encoded, mandatory, no NULLs                                     1.00    100.3±0.85µs        ? ?/sec    1.10    110.0±0.70µs        ? ?/sec
arrow_array_reader/UInt16Array/dictionary encoded, optional, half NULLs                                    1.00    174.3±2.58µs        ? ?/sec    1.03    179.3±3.15µs        ? ?/sec
arrow_array_reader/UInt16Array/dictionary encoded, optional, no NULLs                                      1.00    104.9±0.80µs        ? ?/sec    1.08    113.4±0.65µs        ? ?/sec
arrow_array_reader/UInt16Array/plain encoded, mandatory, no NULLs                                          1.00     36.7±0.27µs        ? ?/sec    1.00     36.9±0.19µs        ? ?/sec
arrow_array_reader/UInt16Array/plain encoded, optional, half NULLs                                         1.00    139.1±1.44µs        ? ?/sec    1.00    139.0±2.22µs        ? ?/sec
arrow_array_reader/UInt16Array/plain encoded, optional, no NULLs                                           1.03     41.3±0.21µs        ? ?/sec    1.00     40.0±0.17µs        ? ?/sec
arrow_array_reader/UInt32Array/binary packed skip, mandatory, no NULLs                                     1.00     82.3±0.53µs        ? ?/sec    1.00     82.2±0.95µs        ? ?/sec
arrow_array_reader/UInt32Array/binary packed skip, optional, half NULLs                                    1.00    102.1±1.68µs        ? ?/sec    1.00    101.9±0.49µs        ? ?/sec
arrow_array_reader/UInt32Array/binary packed skip, optional, no NULLs                                      1.01     85.1±2.47µs        ? ?/sec    1.00     84.2±1.34µs        ? ?/sec
arrow_array_reader/UInt32Array/binary packed, mandatory, no NULLs                                          1.00    107.2±1.41µs        ? ?/sec    1.00    107.0±0.71µs        ? ?/sec
arrow_array_reader/UInt32Array/binary packed, optional, half NULLs                                         1.00    166.5±0.69µs        ? ?/sec    1.00    167.1±3.58µs        ? ?/sec
arrow_array_reader/UInt32Array/binary packed, optional, no NULLs                                           1.01    111.6±0.56µs        ? ?/sec    1.00    110.7±3.37µs        ? ?/sec
arrow_array_reader/UInt32Array/byte_stream_split encoded, mandatory, no NULLs                              1.09     23.9±0.76µs        ? ?/sec    1.00     22.0±0.46µs        ? ?/sec
arrow_array_reader/UInt32Array/byte_stream_split encoded, optional, half NULLs                             1.00    121.0±0.67µs        ? ?/sec    1.01    121.9±1.88µs        ? ?/sec
arrow_array_reader/UInt32Array/byte_stream_split encoded, optional, no NULLs                               1.11     28.3±0.42µs        ? ?/sec    1.00     25.6±0.48µs        ? ?/sec
arrow_array_reader/UInt32Array/dictionary encoded, mandatory, no NULLs                                     1.00     82.7±1.55µs        ? ?/sec    1.10     91.2±1.15µs        ? ?/sec
arrow_array_reader/UInt32Array/dictionary encoded, optional, half NULLs                                    1.00    155.0±1.34µs        ? ?/sec    1.03    160.0±4.44µs        ? ?/sec
arrow_array_reader/UInt32Array/dictionary encoded, optional, no NULLs                                      1.00     87.2±1.20µs        ? ?/sec    1.09     95.2±2.99µs        ? ?/sec
arrow_array_reader/UInt32Array/plain encoded, mandatory, no NULLs                                          1.03     18.7±0.73µs        ? ?/sec    1.00     18.1±0.66µs        ? ?/sec
arrow_array_reader/UInt32Array/plain encoded, optional, half NULLs                                         1.00    119.5±1.79µs        ? ?/sec    1.00    119.4±1.98µs        ? ?/sec
arrow_array_reader/UInt32Array/plain encoded, optional, no NULLs                                           1.10     23.9±1.33µs        ? ?/sec    1.00     21.8±0.57µs        ? ?/sec
arrow_array_reader/UInt64Array/binary packed skip, mandatory, no NULLs                                     1.00     79.9±3.82µs        ? ?/sec    1.00     79.7±0.34µs        ? ?/sec
arrow_array_reader/UInt64Array/binary packed skip, optional, half NULLs                                    1.00     90.5±4.25µs        ? ?/sec    1.00     90.5±0.60µs        ? ?/sec
arrow_array_reader/UInt64Array/binary packed skip, optional, no NULLs                                      1.00     82.2±1.63µs        ? ?/sec    1.00     82.6±1.17µs        ? ?/sec
arrow_array_reader/UInt64Array/binary packed, mandatory, no NULLs                                          1.00    107.0±1.39µs        ? ?/sec    1.01    108.1±1.73µs        ? ?/sec
arrow_array_reader/UInt64Array/binary packed, optional, half NULLs                                         1.00    149.3±0.78µs        ? ?/sec    1.03    153.1±0.95µs        ? ?/sec
arrow_array_reader/UInt64Array/binary packed, optional, no NULLs                                           1.00    111.5±1.07µs        ? ?/sec    1.01    112.3±1.62µs        ? ?/sec
arrow_array_reader/UInt64Array/byte_stream_split encoded, mandatory, no NULLs                              1.00    145.7±1.22µs        ? ?/sec    1.01    147.0±1.26µs        ? ?/sec
arrow_array_reader/UInt64Array/byte_stream_split encoded, optional, half NULLs                             1.00    167.7±1.72µs        ? ?/sec    1.01    169.7±2.47µs        ? ?/sec
arrow_array_reader/UInt64Array/byte_stream_split encoded, optional, no NULLs                               1.00    151.1±0.88µs        ? ?/sec    1.00    150.4±0.59µs        ? ?/sec
arrow_array_reader/UInt64Array/dictionary encoded, mandatory, no NULLs                                     1.09     96.7±2.09µs        ? ?/sec    1.00     88.5±0.49µs        ? ?/sec
arrow_array_reader/UInt64Array/dictionary encoded, optional, half NULLs                                    1.03    140.9±3.27µs        ? ?/sec    1.00    136.5±1.15µs        ? ?/sec
arrow_array_reader/UInt64Array/dictionary encoded, optional, no NULLs                                      1.11    101.5±0.73µs        ? ?/sec    1.00     91.0±1.43µs        ? ?/sec
arrow_array_reader/UInt64Array/plain encoded, mandatory, no NULLs                                          1.00     38.0±0.61µs        ? ?/sec    1.16     44.0±3.52µs        ? ?/sec
arrow_array_reader/UInt64Array/plain encoded, optional, half NULLs                                         1.00    110.4±1.46µs        ? ?/sec    1.02    112.6±0.92µs        ? ?/sec
arrow_array_reader/UInt64Array/plain encoded, optional, no NULLs                                           1.00     43.5±0.67µs        ? ?/sec    1.11     48.2±3.53µs        ? ?/sec
arrow_array_reader/UInt8Array/binary packed skip, mandatory, no NULLs                                      1.01     88.7±3.26µs        ? ?/sec    1.00     87.9±1.11µs        ? ?/sec
arrow_array_reader/UInt8Array/binary packed skip, optional, half NULLs                                     1.00    106.8±0.62µs        ? ?/sec    1.00    106.8±2.89µs        ? ?/sec
arrow_array_reader/UInt8Array/binary packed skip, optional, no NULLs                                       1.01     90.3±0.51µs        ? ?/sec    1.00     89.5±0.80µs        ? ?/sec
arrow_array_reader/UInt8Array/binary packed, mandatory, no NULLs                                           1.00    118.0±1.25µs        ? ?/sec    1.00    117.7±1.15µs        ? ?/sec
arrow_array_reader/UInt8Array/binary packed, optional, half NULLs                                          1.01    177.5±2.73µs        ? ?/sec    1.00    176.2±1.51µs        ? ?/sec
arrow_array_reader/UInt8Array/binary packed, optional, no NULLs                                            1.01    122.6±2.68µs        ? ?/sec    1.00    121.0±1.38µs        ? ?/sec
arrow_array_reader/UInt8Array/byte_stream_split encoded, mandatory, no NULLs                               1.00     34.6±0.42µs        ? ?/sec    1.00     34.6±0.25µs        ? ?/sec
arrow_array_reader/UInt8Array/byte_stream_split encoded, optional, half NULLs                              1.00    132.8±2.59µs        ? ?/sec    1.01    133.7±1.32µs        ? ?/sec
arrow_array_reader/UInt8Array/byte_stream_split encoded, optional, no NULLs                                1.03     38.8±0.25µs        ? ?/sec    1.00     37.6±0.23µs        ? ?/sec
arrow_array_reader/UInt8Array/dictionary encoded, mandatory, no NULLs                                      1.00     93.0±0.30µs        ? ?/sec    1.10    102.3±1.26µs        ? ?/sec
arrow_array_reader/UInt8Array/dictionary encoded, optional, half NULLs                                     1.00    167.2±2.39µs        ? ?/sec    1.03    171.6±5.20µs        ? ?/sec
arrow_array_reader/UInt8Array/dictionary encoded, optional, no NULLs                                       1.00     97.7±0.82µs        ? ?/sec    1.08    105.4±0.74µs        ? ?/sec
arrow_array_reader/UInt8Array/plain encoded, mandatory, no NULLs                                           1.00     28.7±0.17µs        ? ?/sec    1.00     28.8±0.17µs        ? ?/sec
arrow_array_reader/UInt8Array/plain encoded, optional, half NULLs                                          1.01    131.6±3.45µs        ? ?/sec    1.00    130.3±1.09µs        ? ?/sec
arrow_array_reader/UInt8Array/plain encoded, optional, no NULLs                                            1.04     33.6±0.61µs        ? ?/sec    1.00     32.4±0.15µs        ? ?/sec
arrow_array_reader/struct/Int32Array/plain encoded, mandatory struct, optional data, half NULLs            1.00    119.0±0.86µs        ? ?/sec    1.03    122.1±1.86µs        ? ?/sec
arrow_array_reader/struct/Int32Array/plain encoded, mandatory struct, optional data, no NULLs              1.00     22.1±0.58µs        ? ?/sec    1.01     22.3±0.66µs        ? ?/sec
arrow_array_reader/struct/Int32Array/plain encoded, optional struct, optional data, half NULLs             1.02    246.5±1.71µs        ? ?/sec    1.00    242.0±3.13µs        ? ?/sec
arrow_array_reader/struct/Int32Array/plain encoded, optional struct, optional data, no NULLs               1.05    122.9±2.89µs        ? ?/sec    1.00    117.5±0.79µs        ? ?/sec

@lyang24 lyang24 changed the title enhance: rle skip decode loop on batch contains all max levels parquet: rle skip decode loop on batch contains all max levels Jan 26, 2026
@lyang24 lyang24 marked this pull request as ready for review January 26, 2026 08:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

parquet Changes to the parquet crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants