Optimize UnalignedBitChunk

[Dandandan]

Which issue does this PR close?

• Closes #NNN.

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

[Dandandan]

run benchmark filter_kernels boolean_kernels arrow_reader arrow_reader_clickbench

[alamb-ghbot]

🤖 Hi @Dandandan, thanks for the request (#9284 (comment)).

scrape_comments.py only supports whitelisted benchmarks.

• Standard: (none)
• Criterion: array_iter, arrow_reader, arrow_reader_clickbench, arrow_reader_row_filter, arrow_statistics, arrow_writer, bitwise_kernel, boolean_kernels, buffer_bit_ops, cast_kernels, coalesce_kernels, comparison_kernels, concatenate_kernel, csv_writer, encoding, filter_kernels, interleave_kernels, json-reader, metadata, row_format, take_kernels, union_array, variant_builder, variant_kernels, variant_validation, view_types, zip_kernels

Please choose one or more of these with run benchmark <name> or run benchmark <name1> <name2>...

[alamb-ghbot]

🤖 ./gh_compare_arrow.sh gh_compare_arrow.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing optimize_unaligned_bitchunk_iterator (b4b71aa) to bd76edd diff
BENCH_NAME=filter_kernels
BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench filter_kernels
BENCH_FILTER=
BENCH_BRANCH_NAME=optimize_unaligned_bitchunk_iterator
Results will be posted here when complete

[alamb-ghbot]

🤖: Benchmark completed

Details

group                                                                         main                                   optimize_unaligned_bitchunk_iterator
-----                                                                         ----                                   ------------------------------------
filter context decimal128 (kept 1/2)                                          1.08     45.0±6.00µs        ? ?/sec    1.00     41.7±1.63µs        ? ?/sec
filter context decimal128 high selectivity (kept 1023/1024)                   1.00     50.6±1.10µs        ? ?/sec    1.02     51.8±0.92µs        ? ?/sec
filter context decimal128 low selectivity (kept 1/1024)                       1.02    232.1±2.99ns        ? ?/sec    1.00    226.4±2.00ns        ? ?/sec
filter context f32 (kept 1/2)                                                 1.00     77.5±0.62µs        ? ?/sec    1.13     87.9±0.61µs        ? ?/sec
filter context f32 high selectivity (kept 1023/1024)                          1.02     10.1±0.41µs        ? ?/sec    1.00      9.9±0.56µs        ? ?/sec
filter context f32 low selectivity (kept 1/1024)                              1.02    427.5±4.68ns        ? ?/sec    1.00    420.7±1.93ns        ? ?/sec
filter context fsb with value length 20 (kept 1/2)                            1.00     60.6±0.23µs        ? ?/sec    1.16     70.5±0.24µs        ? ?/sec
filter context fsb with value length 20 high selectivity (kept 1023/1024)     1.00     60.6±0.41µs        ? ?/sec    1.17     70.6±0.48µs        ? ?/sec
filter context fsb with value length 20 low selectivity (kept 1/1024)         1.00     60.6±0.26µs        ? ?/sec    1.17     70.6±0.25µs        ? ?/sec
filter context fsb with value length 5 (kept 1/2)                             1.00     60.6±0.54µs        ? ?/sec    1.16     70.5±0.22µs        ? ?/sec
filter context fsb with value length 5 high selectivity (kept 1023/1024)      1.00     60.6±0.48µs        ? ?/sec    1.17     70.6±0.87µs        ? ?/sec
filter context fsb with value length 5 low selectivity (kept 1/1024)          1.00     60.6±0.49µs        ? ?/sec    1.17     70.7±0.43µs        ? ?/sec
filter context fsb with value length 50 (kept 1/2)                            1.00     60.6±0.32µs        ? ?/sec    1.17     70.7±0.52µs        ? ?/sec
filter context fsb with value length 50 high selectivity (kept 1023/1024)     1.00     60.6±0.33µs        ? ?/sec    1.17     70.7±0.57µs        ? ?/sec
filter context fsb with value length 50 low selectivity (kept 1/1024)         1.00     60.7±0.74µs        ? ?/sec    1.17     70.7±0.88µs        ? ?/sec
filter context i32 (kept 1/2)                                                 1.01     16.7±0.10µs        ? ?/sec    1.00     16.5±0.14µs        ? ?/sec
filter context i32 high selectivity (kept 1023/1024)                          1.09      6.8±0.41µs        ? ?/sec    1.00      6.2±0.34µs        ? ?/sec
filter context i32 low selectivity (kept 1/1024)                              1.00    219.3±1.72ns        ? ?/sec    1.01    221.9±1.43ns        ? ?/sec
filter context i32 w NULLs (kept 1/2)                                         1.00     77.7±0.87µs        ? ?/sec    1.13     87.7±1.03µs        ? ?/sec
filter context i32 w NULLs high selectivity (kept 1023/1024)                  1.01     10.3±0.41µs        ? ?/sec    1.00     10.2±0.48µs        ? ?/sec
filter context i32 w NULLs low selectivity (kept 1/1024)                      1.01   423.6±10.73ns        ? ?/sec    1.00    418.9±4.40ns        ? ?/sec
filter context mixed string view (kept 1/2)                                   1.00    104.1±4.37µs        ? ?/sec    1.11    115.2±5.03µs        ? ?/sec
filter context mixed string view high selectivity (kept 1023/1024)            1.00     55.2±1.16µs        ? ?/sec    1.01     55.4±1.38µs        ? ?/sec
filter context mixed string view low selectivity (kept 1/1024)                1.00    605.9±4.53ns        ? ?/sec    1.02    616.7±9.10ns        ? ?/sec
filter context short string view (kept 1/2)                                   1.00    106.7±6.58µs        ? ?/sec    1.14    122.0±6.87µs        ? ?/sec
filter context short string view high selectivity (kept 1023/1024)            1.00     53.8±1.46µs        ? ?/sec    1.03     55.3±1.01µs        ? ?/sec
filter context short string view low selectivity (kept 1/1024)                1.01   475.7±18.16ns        ? ?/sec    1.00    472.6±2.53ns        ? ?/sec
filter context string (kept 1/2)                                              1.00    578.8±7.31µs        ? ?/sec    1.04   603.4±15.56µs        ? ?/sec
filter context string dictionary (kept 1/2)                                   1.00     16.9±0.31µs        ? ?/sec    1.04     17.5±0.71µs        ? ?/sec
filter context string dictionary high selectivity (kept 1023/1024)            1.03      7.2±0.51µs        ? ?/sec    1.00      7.0±0.48µs        ? ?/sec
filter context string dictionary low selectivity (kept 1/1024)                1.00    637.7±4.22ns        ? ?/sec    1.00    637.0±3.36ns        ? ?/sec
filter context string dictionary w NULLs (kept 1/2)                           1.00     78.5±1.31µs        ? ?/sec    1.13     88.5±0.71µs        ? ?/sec
filter context string dictionary w NULLs high selectivity (kept 1023/1024)    1.00     10.5±0.40µs        ? ?/sec    1.03     10.8±0.58µs        ? ?/sec
filter context string dictionary w NULLs low selectivity (kept 1/1024)        1.01    865.4±5.85ns        ? ?/sec    1.00   856.5±53.29ns        ? ?/sec
filter context string high selectivity (kept 1023/1024)                       1.00   634.1±22.32µs        ? ?/sec    1.02   646.8±20.49µs        ? ?/sec
filter context string low selectivity (kept 1/1024)                           1.00    909.3±3.51ns        ? ?/sec    1.08   985.7±12.71ns        ? ?/sec
filter context u8 (kept 1/2)                                                  1.00     15.0±0.10µs        ? ?/sec    1.00     15.0±0.15µs        ? ?/sec
filter context u8 high selectivity (kept 1023/1024)                           1.09  1997.8±17.80ns        ? ?/sec    1.00  1836.2±11.64ns        ? ?/sec
filter context u8 low selectivity (kept 1/1024)                               1.01    215.2±4.55ns        ? ?/sec    1.00    213.9±0.83ns        ? ?/sec
filter context u8 w NULLs (kept 1/2)                                          1.00     76.0±0.63µs        ? ?/sec    1.13     85.9±0.39µs        ? ?/sec
filter context u8 w NULLs high selectivity (kept 1023/1024)                   1.01      5.1±0.04µs        ? ?/sec    1.00      5.0±0.06µs        ? ?/sec
filter context u8 w NULLs low selectivity (kept 1/1024)                       1.00    418.1±4.15ns        ? ?/sec    1.00    417.6±6.57ns        ? ?/sec
filter decimal128 (kept 1/2)                                                  1.11     51.1±5.46µs        ? ?/sec    1.00     45.8±4.29µs        ? ?/sec
filter decimal128 high selectivity (kept 1023/1024)                           1.04     55.2±0.82µs        ? ?/sec    1.00     53.3±1.33µs        ? ?/sec
filter decimal128 low selectivity (kept 1/1024)                               1.91      3.2±0.03µs        ? ?/sec    1.00   1658.8±9.53ns        ? ?/sec
filter f32 (kept 1/2)                                                         1.12    141.2±1.65µs        ? ?/sec    1.00    126.2±0.75µs        ? ?/sec
filter fsb with value length 20 (kept 1/2)                                    1.09    137.4±2.22µs        ? ?/sec    1.00    126.4±0.89µs        ? ?/sec
filter fsb with value length 20 high selectivity (kept 1023/1024)             1.00     71.2±1.94µs        ? ?/sec    1.01     72.0±1.75µs        ? ?/sec
filter fsb with value length 20 low selectivity (kept 1/1024)                 1.80      3.5±0.22µs        ? ?/sec    1.00  1944.9±29.22ns        ? ?/sec
filter fsb with value length 5 (kept 1/2)                                     1.08    142.1±0.76µs        ? ?/sec    1.00    132.1±1.80µs        ? ?/sec
filter fsb with value length 5 high selectivity (kept 1023/1024)              1.12     11.7±0.61µs        ? ?/sec    1.00     10.5±0.56µs        ? ?/sec
filter fsb with value length 5 low selectivity (kept 1/1024)                  1.89      3.4±0.03µs        ? ?/sec    1.00  1809.3±13.79ns        ? ?/sec
filter fsb with value length 50 (kept 1/2)                                    1.13   170.2±17.54µs        ? ?/sec    1.00   150.5±10.19µs        ? ?/sec
filter fsb with value length 50 high selectivity (kept 1023/1024)             1.00    208.9±9.23µs        ? ?/sec    1.00    209.3±6.50µs        ? ?/sec
filter fsb with value length 50 low selectivity (kept 1/1024)                 1.81      3.4±0.04µs        ? ?/sec    1.00  1906.0±17.68ns        ? ?/sec
filter i32 (kept 1/2)                                                         1.24     53.8±1.00µs        ? ?/sec    1.00     43.5±0.36µs        ? ?/sec
filter i32 high selectivity (kept 1023/1024)                                  1.03      9.1±0.55µs        ? ?/sec    1.00      8.7±0.51µs        ? ?/sec
filter i32 low selectivity (kept 1/1024)                                      1.98      3.0±0.02µs        ? ?/sec    1.00  1499.1±11.19ns        ? ?/sec
filter optimize (kept 1/2)                                                    1.01     35.9±0.21µs        ? ?/sec    1.00     35.6±0.24µs        ? ?/sec
filter optimize high selectivity (kept 1023/1024)                             1.19      2.7±0.02µs        ? ?/sec    1.00      2.3±0.02µs        ? ?/sec
filter optimize low selectivity (kept 1/1024)                                 1.33  1793.3±24.31ns        ? ?/sec    1.00  1349.2±14.95ns        ? ?/sec
filter run array (kept 1/2)                                                   1.00    381.6±5.29µs        ? ?/sec    1.21    460.2±4.38µs        ? ?/sec
filter run array high selectivity (kept 1023/1024)                            1.00    395.8±3.91µs        ? ?/sec    1.25    496.0±1.61µs        ? ?/sec
filter run array low selectivity (kept 1/1024)                                1.00    284.4±4.80µs        ? ?/sec    1.34    380.3±3.07µs        ? ?/sec
filter single record batch                                                    1.19     38.5±0.50µs        ? ?/sec    1.00     32.2±0.24µs        ? ?/sec
filter u8 (kept 1/2)                                                          1.12     36.1±0.31µs        ? ?/sec    1.00     32.2±0.12µs        ? ?/sec
filter u8 high selectivity (kept 1023/1024)                                   1.08      3.8±0.03µs        ? ?/sec    1.00      3.5±0.02µs        ? ?/sec
filter u8 low selectivity (kept 1/1024)                                       1.97      3.0±0.03µs        ? ?/sec    1.00  1501.2±11.78ns        ? ?/sec

[alamb-ghbot]

🤖 ./gh_compare_arrow.sh gh_compare_arrow.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing optimize_unaligned_bitchunk_iterator (b4b71aa) to bd76edd diff
BENCH_NAME=boolean_kernels
BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench boolean_kernels
BENCH_FILTER=
BENCH_BRANCH_NAME=optimize_unaligned_bitchunk_iterator
Results will be posted here when complete

[alamb-ghbot]

🤖: Benchmark completed

Details

group            main                                   optimize_unaligned_bitchunk_iterator
-----            ----                                   ------------------------------------
and              1.00    208.4±2.17ns        ? ?/sec    1.00    208.1±3.03ns        ? ?/sec
and_sliced_1     1.00   1097.7±9.34ns        ? ?/sec    1.19  1305.8±16.47ns        ? ?/sec
and_sliced_24    1.00    246.5±0.46ns        ? ?/sec    1.30    319.7±1.06ns        ? ?/sec
not              1.00    145.9±2.30ns        ? ?/sec    1.34    195.2±1.31ns        ? ?/sec
not_slice_24     1.00    194.5±1.26ns        ? ?/sec    1.28    249.8±1.61ns        ? ?/sec
not_sliced_1     2.48   620.6±10.46ns        ? ?/sec    1.00    250.1±4.67ns        ? ?/sec
or               1.00    197.4±1.76ns        ? ?/sec    1.18    232.5±2.72ns        ? ?/sec
or_sliced_1      1.00  1099.7±10.23ns        ? ?/sec    1.25  1377.8±28.46ns        ? ?/sec
or_sliced_24     1.00    248.3±1.25ns        ? ?/sec    1.30    321.8±2.49ns        ? ?/sec

[alamb-ghbot]

🤖 ./gh_compare_arrow.sh gh_compare_arrow.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing optimize_unaligned_bitchunk_iterator (b4b71aa) to bd76edd diff
BENCH_NAME=arrow_reader
BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench arrow_reader
BENCH_FILTER=
BENCH_BRANCH_NAME=optimize_unaligned_bitchunk_iterator
Results will be posted here when complete