jayzhan211 commented on PR #26105: URL: https://github.com/apache/datafusion/pull/26105#issuecomment-6246544096
@adriangb measured both on this branch (release, one batch, per call): | batch | `get_record_batch_memory_size` | sum of `get_array_memory_size` | |---|---:|---:| | int64 ×4, 8192 rows | 24 ns | 3.3 ns | | int64 ×64, 8192 rows | 911 ns | 61 ns | | utf8view ×4, 8192 rows | 230 ns | 6.8 ns | | struct 35×20, 100 rows | 11.7 µs | 0.83 µs | Making `output_bytes` a Dev/verbose metric wouldn't remove the cost on its own: `MetricType` only filters what `AnalyzeExec` displays, and `record_output` still runs for every batch. Once the computation is a plain sum over columns, a config flag doesn't seem worth the extra option to me, but I'm happy to defer to you and @minhh-nguyen. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
