andygrove opened a new pull request, #5032: URL: https://github.com/apache/datafusion-comet/pull/5032
## Which issue does this PR close? Closes #. ## Rationale for this change `translate` (StringTranslate) and `to_csv` (StructsToCsv) have native implementations that are incompatible with Spark by default, so today the whole enclosing operator falls back to Spark unless allowIncompatible is set. Routing them through the JVM codegen dispatcher instead keeps the operator on the native pipeline while producing bit-exact Spark results, and preserves the faster native path as an opt-in. ## What changes are included in this PR? - Route `translate` (StringTranslate) and `to_csv` (StructsToCsv) through the JVM codegen dispatcher by default, using the same `NativeOptInAvailable` pattern as `to_json` and `split`. The dispatcher runs Spark's own generated code (bit-exact) while keeping the operator on the native pipeline; the native path stays available via `allowIncompatible`. - Update the docs to note the codegen-dispatch default for `to_csv`. ## How are these changes tested? - Update the `string_translate` SQL test (previously asserting fallback) to verify native codegen dispatch. - Add a `to_csv` SQL file test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
