Thanks Andy for starting this discussion! I agree we should prioritize the iceberg-rust path as the long term default for Comet/Iceberg scans, while it's still valuable to keep the Iceberg Java integration as an option, especially for users who need JVM-specific features. I'm hoping we can get consensus on a path to resolve the shading issues (e.g. the approach proposed in github.com/apache/iceberg/pull/13786) so the java option can move forward.
Thanks, Huaxin On Wed, Dec 17, 2025 at 12:23 PM Shawn Chang <[email protected]> wrote: > Thanks Andy for starting this thread! > > I've replied to the issue above but am posting here as well for better > visibility: > > I’m leaning toward prioritizing the iceberg-rust path to avoid the shading > and circular dependency headaches of the Java approach. Easier maintenance > typically lowers the entry barrier and encourages more community > involvement, which is important for the project’s long-term health. > > It also makes sense to keep the Iceberg Java integration available as an > experimental option for users who need its richer JVM-specific features > while Rust becomes the long-term default. We don’t want to block > contributions from anyone who truly needs the Java path. > > Best, > Shawn > > On Wed, Dec 17, 2025 at 11:00 AM Andy Grove <[email protected]> wrote: > >> Hi, >> >> There is a discussion in the DataFusion Comet community about the future >> direction of our Iceberg support [1]. For example, should we continue the >> efforts to integrate with the Iceberg Java project or focus more on the >> iceberg-rust project. >> >> It would be great to also get input from the Iceberg community. >> >> Thanks, >> >> Andy. >> >> [1] https://github.com/apache/datafusion-comet/issues/2921 >> >
