Skip to content

Add FFI query planner support #1677

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
timsaucer wants to merge 4 commits into from
Open

Add FFI query planner support #1677

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
9ba147c
Add FFI query planner support
timsaucer Jul 31, 2026
47aea03
Add three-library FFI planner example
timsaucer Jul 31, 2026
05ab5da
Update FFI query planner integration
timsaucer Aug 7, 2026
27147f3
add rat
timsaucer Aug 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 14 additions & 2 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ jobs:
manylinux: "2_28"

# FFI test wheel only needs to be built once per platform; gate to abi3.
- name: Build FFI test library
- name: Build FFI provider test library
if: matrix.python-tag == 'abi3'
uses: PyO3/maturin-action@v1
with:
Expand All @@ -196,6 +196,16 @@ jobs:
args: --out dist
rustup-components: rust-std

- name: Build FFI query planner test library
if: matrix.python-tag == 'abi3'
uses: PyO3/maturin-action@v1
with:
target: x86_64-unknown-linux-gnu
manylinux: "2_28"
working-directory: examples/datafusion-ffi-query-planner-example
args: --out dist
rustup-components: rust-std
Comment on lines +199 to +207

@timsaucer timsaucer Aug 7, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In order to prove that the 3 library approach works where we have different codecs and different execution plans provided, we are adding a second test library. This way we can make sure there is no accidental ability to reach into a foreign code block.


- name: Archive wheels
uses: actions/upload-artifact@v7
with:
Expand All @@ -207,7 +217,9 @@ jobs:
uses: actions/upload-artifact@v7
with:
name: test-ffi-manylinux-x86_64
path: examples/datafusion-ffi-example/dist/*
path: |
examples/datafusion-ffi-example/dist/*
examples/datafusion-ffi-query-planner-example/dist/*

# ============================================
# Build - Linux ARM64
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,8 @@ jobs:
run: |
cd examples/datafusion-ffi-example
uv run --no-project pytest python/tests/_test*.py
cd ../datafusion-ffi-query-planner-example
uv run --no-project pytest python/tests/_test*.py

- name: Run tpchgen-cli to create 1 Gb dataset
if: matrix.wheel-tag == 'abi3'
Expand Down
Loading
Loading