parthchandra commented on PR #1138: URL: https://github.com/apache/datafusion-comet/pull/1138#issuecomment-2528772645
Not really. They are pretty much independent. The difference is that POC 1 uses DF ParquetExec operator directly. POC 2 uses arrow reader to replace the existing native column readers with arrow column readers and also replaces the jvm file reader with the arrow native file reader. Iceberg integration uses the column readers directly which is why we are doing it this way. At the moment the code paths are completely independent and have no code overlap. I'm looking at using DF under the covers instead of arrow directly so we can use the schema adapter code written in POC 1 and we'll be able to reuse some of the work in POC1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
