andygrove opened a new pull request, #4885: URL: https://github.com/apache/datafusion-comet/pull/4885
> ⚠️ **Autonomously generated by an LLM.** This PR was created end-to-end by an automated agent with no human author. It has passed automated correctness (unit tests + differential fuzz) and micro-benchmark gates, but **requires full human review**. Do not merge without expert scrutiny. ## What changed Optimizes `spark_base64` in the native `spark-expr` crate. Reuse scratch String buffers for base64 encode + line-wrap and build directly into a preallocated StringBuilder, eliminating two per-row heap allocations in the array path. ## Evidence - Correctness: unit tests + seeded differential fuzz (bit-identical Arrow output vs `main`). - Benchmark (criterion, Rust-only): see numbers below; gate required at least one benchmark ≥ 5.0% faster (non-overlapping confidence intervals) and no benchmark meaningfully slower. ``` spark_base64_ long, chunked: 21.073% faster (base 882215ns -> cand 696308ns) spark_base64_ short, chunked: 11.932% faster (base 216853ns -> cand 190979ns) spark_base64_ short, unchunked: 12.069% faster (base 215962ns -> cand 189896ns) spark_base64_ long, unchunked: 11.43% faster (base 627480ns -> cand 555761ns) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
