Skip to content

Lazily untranspose DeltaArray validity and depend on fastlanes bit transpose code#8725

Open
robert3005 wants to merge 5 commits into
developfrom
rk/transposedvalidity
Open

Lazily untranspose DeltaArray validity and depend on fastlanes bit transpose code#8725
robert3005 wants to merge 5 commits into
developfrom
rk/transposedvalidity

Conversation

@robert3005

Copy link
Copy Markdown
Contributor

Delta array unnecessarily untraposes whole validity whenever it's queried. This
pr avoids this by laziliy wrapping it in a TransposedBool array that does
untranspose when necessary.

While we're at it we remove the bit transpose logic that has been moved to
fastlanes repo

@robert3005 robert3005 added the changelog/feature A new feature label Jul 12, 2026
@codspeed-hq

codspeed-hq Bot commented Jul 12, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

⚡ 6 improved benchmarks
❌ 5 regressed benchmarks
✅ 1645 untouched benchmarks
⏩ 50 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation chunked_varbinview_into_canonical[(100, 100)] 271.5 µs 306.6 µs -11.46%
Simulation encode_varbin[(1000, 8)] 138.9 µs 155.5 µs -10.72%
Simulation encode_varbin[(1000, 4)] 138.7 µs 154.5 µs -10.24%
Simulation encode_varbin[(1000, 32)] 143.9 µs 160.3 µs -10.23%
Simulation encode_varbin[(1000, 2)] 138.3 µs 154 µs -10.21%
Simulation chunked_bool_canonical_into[(1000, 10)] 26.7 µs 16.4 µs +62.36%
Simulation bitwise_not_vortex_buffer_mut[128] 244.4 ns 186.1 ns +31.34%
Simulation bitwise_not_vortex_buffer_mut[1024] 304.7 ns 246.4 ns +23.68%
Simulation chunked_varbinview_into_canonical[(1000, 10)] 205.9 µs 169.5 µs +21.45%
Simulation bitwise_not_vortex_buffer_mut[2048] 398.6 ns 340.3 ns +17.14%
Simulation eq_i64_constant 298.5 µs 268.7 µs +11.1%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing rk/transposedvalidity (dd0e1d9) with develop (1d7951a)

Open in CodSpeed

Footnotes

  1. 50 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@robert3005 robert3005 force-pushed the rk/transposedvalidity branch from 26bff75 to 0fd420f Compare July 12, 2026 16:15
Comment thread encodings/fastlanes/benches/bit_transpose.rs Outdated
Comment thread encodings/fastlanes/benches/bit_transpose.rs Outdated
Comment thread encodings/fastlanes/src/bit_transpose/validity.rs Outdated
Comment thread encodings/fastlanes/src/bit_transpose/validity.rs Outdated
Comment on lines +143 to +146
struct TransposedBoolMetadata {
#[prost(uint32, tag = "1")]
offset: u32,
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The transport is width dependent no?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand what you mean

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a lot of shared logic with out blocked arrays?

@robert3005 robert3005 force-pushed the rk/transposedvalidity branch from 0fd420f to 774f405 Compare July 13, 2026 11:49
…anspose code

Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
@robert3005 robert3005 force-pushed the rk/transposedvalidity branch from 774f405 to d830e48 Compare July 13, 2026 14:57
Signed-off-by: Robert Kruszewski <github@robertk.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants