Dandandan commented on issue #13831: URL: https://github.com/apache/datafusion/issues/13831#issuecomment-2554348568
> Perhaps we just manually call `Vec::reserve(new_amount)` to make it behave as expected? I believe that won't change the allocation behavior of `Vec`? I think it's important to keep the exponential growing behavior to maintain performance / runtime complexity. I think we have to: * fix / improve the memory tracking to record bytes used by underlying `Vec`s capacity. * We could try to improve storage by keeping them in large chunks rather than in a single `Vec` (in order to keep memory usage lower), this will be however be performance sensitive. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
