Re: [DISCUSS][Rust] Frequency of upgrading minimum supported rust version.

Thanks xxchan’s summary, we also reached consensus about msrv: 1 At least three months from latest rust release 2 Update msrv when we release iceberg-rust On Fri, Feb 28, 2025 at 00:16 xxchan wrote: > Conclusion in today's meeting: > - Test against latest version in CI (by updating Cargo.lock

Re: [DISCUSS][Rust] Frequency of upgrading minimum supported rust version.

Conclusion in today's meeting: - Test against latest version in CI (by updating Cargo.lock with dependabot) - Have a wider range of versions support, to allow users to choose their dep version (by not updating Cargo.toml too often). Perhaps need to add a CI to test against oldest version. On Thu,

Re: [DISCUSS][Rust] Frequency of upgrading minimum supported rust version.

Hi, Xuanwo: If we detect that iceberg is not compatible with lib A 0.1.10, then we could know that in advance, and we could figure out ways to fix it. Users have several choices, like using Cargo.lock to pin the version used. We could have a discussion in this week's community sync. On Tue, Feb

Re: [DISCUSS][Rust] Frequency of upgrading minimum supported rust version.

Hi, renjie > My point is still the same, if iceberg uses lib A 0.1.9, and the 0.1.10 > version of lib A doesn't work with iceberg 0.3, the user of iceberg will face > either of: > 1. Forced to use the old version of lib A, without enjoying the performance / > bug fix of the new version of lib A

Re: [DISCUSS][Rust] Frequency of upgrading minimum supported rust version.

Hi, Xuanwo: Thanks for pointing that out, and you're right about the version used. However, if iceberg depends on lib A 0.1.10, but app B depends on lib A > 0.1.9, users will finally get: > - iceberg 0.3 > - lib A 0.1.10 (the biggest if all lib A 0.1.X) My point is still the same, if iceberg us

Re: [DISCUSS][Rust] Frequency of upgrading minimum supported rust version.

Hi, renjie > 1. Iceberg 0.3 dependents library a 0.1 > 2. Application b dependents on iceberg 0.3 and library a 0.2 > 3. Library a 0.2 doesn't work with iceberg 0.3 > 4. Application b crashed in production. This example can't happen in rust. In rust, every version within 0.X is considered a brea

Re: [DISCUSS][Rust] Frequency of upgrading minimum supported rust version.

Thanks to xxchan's explanation, it seems that we need to resolve the dependency version first. 2.1 Always use the latest dependency version in Cargo.toml, and force > downstream applications to upgrade transitive dependencies at the same time. > Benefits: let downstream enjoy latest bug fixes and

Re: [DISCUSS][Rust] Frequency of upgrading minimum supported rust version.

> 2.2 Do not upgrade Cargo.toml, and let the application be responsible for the > transitive dependencies' version. Instead, we upgrade in Cargo.lock which > means "we've tested the latest version". I prefer this approach. I believe we should disable dependabot from upgrading our patch versions

Re: [DISCUSS][Rust] Frequency of upgrading minimum supported rust version.

Hi Renjie, Yes you are right that `datafusion-iceberg` will not break. I somehow mistakenly thought that we cannot compile `datafusion` with a newer rust toolchain... Sorry for the mistake. Regarding the motivations > 1. Use new(not latest) features in stable rust as rust is still evolving. In

Re: [DISCUSS][Rust] Frequency of upgrading minimum supported rust version.

Hi, xxchan: In fact, datafusion is not a consumer of iceberg-rust, in fact, `datafusion-iceberg` is a consumer of both `iceberg` and `datafusion`, which is maintained in `iceberg-rust` repo, so a more aggressive msrv policy will not break `datafusion-iceberg`. On Sun, Feb 23, 2025 at 8:40 PM xxc

Re: [DISCUSS][Rust] Frequency of upgrading minimum supported rust version.

Hi, I want to explain why we need to upgrade msrv from time to time: 1. Use new(not latest) features in stable rust as rust is still evolving. 2. Use newer versions of iceberg-rust's dependencies, which may include important bug fixes and performance improvements. I've seen dependent bot auto upg

Re: [DISCUSS][Rust] Frequency of upgrading minimum supported rust version.

Hi, To clarify, if we want to adopt the "at least three months" policy, we will need to wait for 3 months before upgrading to 1.85. And I believe we don't want to break Datafusion integration, so we will have to use the "at least six months" policy (or more conservative). xxchan On Sun, Feb 23,

Re: [DISCUSS][Rust] Frequency of upgrading minimum supported rust version.

Hi, As Xuanwo mentions the "at least three months," policy looks good to me. Also, it's time to upgrade to Rust 1.85 (the first version to support Edition 2024!). See: https://github.com/apache/iceberg-rust/pull/987

Re: [DISCUSS][Rust] Frequency of upgrading minimum supported rust version.

Hi, I think we can just follow Datafusion's policy, which is a consumer and relatively conservative (4 minor versions, or ~6 month ) https://github.com/apache/datafusion?tab=readme-ov-file#rust-version-compatibility-policy xxchan On Sat, Feb 22, 2025 at 04:21 Christian Thiel wrote: > Dear both

Re: [DISCUSS][Rust] Frequency of upgrading minimum supported rust version.

Dear both, Thanks for bringing this up! As iceberg-rust is first and foremost a library crate (despite us committing the Cargo.lock) I would try to not update too quickly. Three months sounds reasonable to me. We probably don't see most of the crate's consumers on crates.io, as they are binaries.

Re: [DISCUSS][Rust] Frequency of upgrading minimum supported rust version.

Personally I prefer a longer gap(like three months) so that we don't need to force users to upgrade to the rust version. But this may not be a big problem in the rust world as rust's release is usually quite stable and backward compatible. On Fri, Feb 21, 2025 at 11:30 AM Renjie Liu wrote: > Hi:

Re: [DISCUSS][Rust] Frequency of upgrading minimum supported rust version.

Hi, renjie Thank you for bringing this up. Most of our users are currently using the latest stable or even nightly Rust, so MSRV itself is not a major concern at the moment. We can upgrade to Rust 1.85 (the first version to support Edition 2024!) today. However, if we do need to establish an M

[DISCUSS][Rust] Frequency of upgrading minimum supported rust version.

Hi: As discussed in this issue , we have landed support for using unstable rust for tooling, while stable rust for publishing libraries. Also we enforced checking of msrv(minimum supported rust version) in our ci. However, there is one thing undet