debugmiller opened a new pull request, #105:
URL: https://github.com/apache/datafusion-iceberg/pull/105
Closes #104
`IS NULL` / `IS NOT NULL` on a struct, list or map column fails the scan,
because iceberg-rust can only bind
predicates on primitive fields. Filters are pushed down as `Inexact`, so the
predicate never needed to reach Iceberg.
In this change, `IcebergDataSourceBuilder::build` splits the predicate into
its top-level conjuncts and keeps those that bind to the
scanned snapshot's schema. In `WHERE id = 1 AND s IS NOT NULL`, `id = 1` is
still pushed. An `OR` with a side that does
not bind is dropped whole. No public API changes.
Tests:
- A SQL test (`nested_null_predicate_pushdown.slt`) creates a table with a
struct column and checks that `IS NULL` and
`IS NOT NULL` on it return the right rows (both fail on `main`), and that
a null test on a field inside the struct
still works.
- A unit test (`test_scan_drops_conjuncts_that_do_not_bind`) builds scans
with predicates that partly bind and checks
that the binding conjuncts are kept, that a predicate with nothing that
binds becomes no predicate, and that an `OR`
with an unbindable side is dropped whole.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
