timsaucer commented on issue #13851: URL: https://github.com/apache/datafusion/issues/13851#issuecomment-2557143735
There is only one point where we have ffi calls that are async, currently. This happens in the record batch stream. So I think what needs to happen is that within the private data for the RBS we also include an `Option<Arc<Runtime>>` and if it exists, we enter the runtime during the async calls. We *do* pass a context. I worry about a possible problem where we could have deadlocks if we have two different runtimes each doing a blocking call. The only place that is immediately jumping out to me as a potential problem spot is `RecordBatchReceiverStreamBuilder`, but I haven't had time to thoroughly look through this. I think the immediate next thing to do is to build up a good reproducible test, ideally that I could include as a unit test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
