alexanderbianchi commented on issue #23: URL: https://github.com/apache/datafusion-iceberg/issues/23#issuecomment-5900977145
Thanks for the write-up! A lot of these primitives overlap with what DFD needs, which is great. In DFD, we’ll maintain our own `DataSourceExec` integration because it’s built on the `WorkUnitFeeds` abstraction. Ideally, we can reuse upstream for everything else. As these APIs become public, I’d like to make sure they also accommodate DFD’s execution model—particularly running writers on workers and collecting their results for a coordinator-side commit. Breaking this into sub-issues would help; I know a few engineers looking to pick up tickets, including myself. I can ask @toutane whether he wants to move [[iceberg-rust#2671](https://github.com/apache/iceberg-rust/pull/2671)](https://github.com/apache/iceberg-rust/pull/2671) here or whether I should carry it forward—we work together. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
