peterxcli opened a new pull request, #5859: URL: https://github.com/apache/datafusion-comet/pull/5859
## Which issue does this PR close? Related to #5485. ## Rationale for this change Spark row consumers of Comet's cache pay for iterator adapters, projection through `ColumnarBatchRow`, and a final row copy on every row, even for `count(*)`. ## What changes are included in this PR? Generate an indexed iterator that reads selected vectors directly into Spark's reusable `UnsafeRow` writer, removing the adapters and extra copy. Preserve owned variable-width values and Spark's interpreted fallback. ## How are these changes tested? Cache and iterator suites pass on Spark 3.5.9 and 4.1.3, covering row ownership, nulls, nested values, sorting, joins, and batch boundaries. The [5M-row benchmark](https://github.com/peterxcli/datafusion-comet/blob/ec2c5b584feda0d72a0b697cb5d44d2ae0a34be3/benchmarks/results/cache-consumer/2026-09-11-row-reader.txt) shows 14–71% lower cached-read time versus main across the tested projections; all 810 measured actions matched uncached answers. Broader reads still trail Spark's own cache. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
