jaystarshot commented on issue #18308: URL: https://github.com/apache/hudi/issues/18308#issuecomment-4112749139
@vinothchandar At Uber, our primary focus is on Presto/Velox (Prestissimo). Since we operate in a multi-query environment, we have concerns about adopting hudi-rs, particularly around memory management. 1) Is it used in production in any other companies? 2) In Prestissimo, memory allocation is tightly controlled at the per-query level in C++, with enforced limits. If new memory is allocated in Rust (hudi-rs), it may not integrate with our existing memory management model, potentially leading to issues such as uncontrolled memory usage or increased risk of worker OOMs. 3) How is the data transfer handled between hudi-rs and velox, is it 0 copy? If it uses velox ABI is velox ABI stable? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
