Hi Minglei, thanks for the proposal. v3 is now closed, so we can't introduce a breaking change like this until v4. We looked into decimal type evolution in v3 and found that due to the way that we currently store lower and upper bounds for decimal values, we can't safely support this in v3 Iceberg manifests. We will need to wait until v4 manifests are introduced with columnar metadata to make this change.
Ryan On Wed, Sep 10, 2025 at 12:28 AM rice Zhang <minglei...@gmail.com> wrote: > Hi Iceberg Community, > > I'd like to propose extending Iceberg's type promotion rules to support > DECIMAL type evolution with scale changes, aligning with the SQL:2011 > standard. > > *Current Limitation* > Currently, Iceberg only supports DECIMAL type promotion when: > - Scale remains the same > - Precision can be increased > > This means DECIMAL(10,2) can evolve to DECIMAL(12,2), but not to > DECIMAL(12,4). > > *Proposed Change* > Allow DECIMAL type evolution when: > 1. Target scale >= source scale > 2. Target precision >= source precision > 3. Integer part capacity is preserved: (target_precision - target_scale) > >= (source_precision - source_scale) > > *Examples* > With this change: > - DECIMAL(10,2) → DECIMAL(12,4) ✓ (integer part: 8 → 8, scale: 2 → 4) > - DECIMAL(10,2) → DECIMAL(15,5) ✓ (integer part: 8 → 10, scale: 2 → 5) > - DECIMAL(10,2) → DECIMAL(10,4) ✗ (integer part: 8 → 6, would lose > integer capacity) > > *Rationale* > 1. SQL:2011 Compliance: This behavior aligns with SQL:2011 standard > expectations > 2. User Experience: Many users coming from traditional databases expect > this type evolution to work > 3. Data Safety: The proposed rules ensure no data loss - existing values > can always be represented in the new > type > 4. Real-world Use Cases: Common scenarios like adding more decimal > precision for currency calculations would > be supported > > *Implementation* > I've created a proof-of-concept implementation: > https://github.com/apache/iceberg/issues/14037 > > *Questions for Discussion* > 1. Should this be part of the spec v3, or wait for a future version? > 2. Are there any backward compatibility concerns we should address? > > Looking forward to your feedback and thoughts on this proposal. > > Best regards, > Minglei >
