Rachelint commented on PR #15591: URL: https://github.com/apache/datafusion/pull/15591#issuecomment-2890634742
> I wonder what happens if we make it more like at least 1 million or 1MiB so the effect on cache-friendliness is smaller? > We could optimize a growing strategy for the first allocated Vec if memory usage / overhead of first block is a concern. I have tried to larger the block size(8 * batch, 16 * batch...), but it seems make slight difference to the performance. So after experiement, I think `single vector + resizing` is efficient enough actually... - It is more efficient for random access - Resizing will only happen a few times, so it is acceptable actually -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
