Re: [DISCUSS] Use iceberg-rust for PyIceberg Bucket Transform

Hi folks, thank you all for your input. It looks like there's a general agreement that this is a good idea. Given that Xuanwo has already put the scaffolding in place for pyiceberg_core , I've put up this PR to implement bucket_transform

Re: [DISCUSS] Use iceberg-rust for PyIceberg Bucket Transform

Hi: Would pushing the bucket transforms into Rust a good exercise to get the > scaffolding in place? To be honest, I don't think bucket transforms alone is not a good starting point, maybe a more general approach would be to provide another set of transform implementations backed by iceberg-rust

Re: [DISCUSS] Use iceberg-rust for PyIceberg Bucket Transform

Hey everyone, In the beginning of PyIceberg, one of the goals was to keep PyIceberg pure Python. At some point, we've added a Cython Avro decoder because of performance reasons, but we still have a pure Python fallback. Today you can still do metadata operating using s3fs without any native code.

Re: [DISCUSS] Use iceberg-rust for PyIceberg Bucket Transform

Hi: Thanks Sung for raising this. Just as Ryan said, I'm also +1 for using more iceberg-rust in pyiceberg. Is it that we would introduce a hard dependency on iceberg-rust? I think so. Is that a risk that could make PyIceberg unusable for some people? I don't > think that would be a problem sin

Re: [DISCUSS] Use iceberg-rust for PyIceberg Bucket Transform

In general, I think the idea of using iceberg-rust more from PyIceberg is great. I think it will be a good path to pushing more things down to native code. What are the trade-offs of doing it this way? Is it that we would introduce a hard dependency on iceberg-rust? Is that a risk that could make