sunchao opened a new pull request, #5845: URL: https://github.com/apache/datafusion-comet/pull/5845
## Which issue does this PR close? Follow-up to #5602 and #5495. ## Rationale for this change When JVM field-name folding fails, the Parquet reader currently logs the failure and switches to Rust Unicode lowercasing. The Unicode tables can differ: JDK 17 keeps `` and `ƛ` distinct, while Rust lowercases the former to the latter. Continuing after a JNI failure can therefore resolve a different physical column instead of surfacing the original error. ## What changes are included in this PR? - Return folding errors through projection planning, schema remapping, nested conversion, and structural-pruning eligibility. Production builds also return an error when a non-ASCII fold needs an uninitialized JVM. - Publish cache misses only after the entire JVM fold succeeds; retain the ASCII and case-sensitive fast paths. - Propagate nested default-value conversion errors instead of inserting the original unconverted literal. - Add injected-failure/cache-recovery and fast-path regressions, plus a nested-default duplicate-field regression. Retain the test-only Unicode substitute for standalone native unit tests without a JVM. This follow-up leaves field-ID matching and calendar integration to their existing contributions. ## How are these changes tested? Passed locally: - `cargo fmt --manifest-path native/Cargo.toml --all -- --check` - `./mvnw spotless:check -Pjdk17` - `git diff --check` - Five folding/cache tests executed from the edited helper source in an isolated Rust harness. The harness substitutes the external error type and no-JVM provider; this validates the local folding/cache logic, not JNI or DataFusion integration. Validation blocked before compilation: - `make`, the new nested-default regression against the unchanged baseline, and workspace Clippy all stop during dependency resolution: the managed registry does not provide upstream's locked `aws-smithy-runtime-api 1.16.0`. - The lockfile is unchanged. Full native tests, the new nested-default regression, and `CometNativeReaderSuite` against a rebuilt library still need to pass. This PR remains draft pending that validation; no failing-before/passing-after integration result is claimed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
