We found an issue with the DynamicSink, and Max is working on it: https://github.com/apache/iceberg/pull/14358 Added to the milestone, and I hope we can get this in and the backport of it to the other Flink versions
huaxin gao <[email protected]> ezt írta (időpont: 2025. okt. 17., P, 8:06): > Hi all, > Here are the candidate PRs for 1.10.1 so far. If anything should be > added, please let me know. I will keep this list updated and share a final > changelog before cutting the RC. > > *Merged:* > > - > > Spark 3.5: Pass format-version when creating a snapshot — #14169 > <https://github.com/apache/iceberg/pull/14169> > - > > Spark 3.4: Pass format-version when creating a snapshot — #14170 > <https://github.com/apache/iceberg/pull/14170> > - > > Spark 4.0: Pass format-version when creating a snapshot in table > migration actions — #14163 > <https://github.com/apache/iceberg/pull/14163> > - > > Fix variant type filtering in ParquetMetricsRowGroupFilter — #14081 > <https://github.com/apache/iceberg/pull/14081> > - > > Core: Fixed the unpartitioned check in replace partitions — #14186 > <https://github.com/apache/iceberg/pull/14186> > - > > API: Required nested fields within optional structs can produce nulls > — #13804 <https://github.com/apache/iceberg/pull/13804> > - > > Flink: Backport #14182 — ensure DynamicCommitter idempotence in the > presence of failures — #14213 > <https://github.com/apache/iceberg/pull/14213> > - > > Flink: Ensure DynamicCommitter idempotence in the presence of failures > — #14182 <https://github.com/apache/iceberg/pull/14182> > - > > API: Detect whether required fields nested within optionals can > produce nulls — #14270 <https://github.com/apache/iceberg/pull/14270> > - > > BigQuery: Add iceberg-bigquery dependency to Spark and Flink build > scripts — #14221 <https://github.com/apache/iceberg/pull/14221> > - > > Data/Parquet: Fix UUID ClassCastException when reading Parquet files > with UUIDs — #14027 <https://github.com/apache/iceberg/pull/14027> > - > > AWS/Build: Exclude logging dependencies from bundle — #14225 > <https://github.com/apache/iceberg/pull/14225> > - > > Core/Spark: Respect custom location providers in SerializableTable > (#12564) — #14280 <https://github.com/apache/iceberg/pull/14280> > > - > > Parquet: Handle NPE for VariantLogicalType in TypeWithSchemaVisitor — > #14261 <https://github.com/apache/iceberg/pull/14261> > > *Open:* > > - > > AWS: Configure builder for reuse of HTTP connection pool in SDK v2 — > #14161 <https://github.com/apache/iceberg/pull/14161> > - > > AWS / S3 Signing: Fix leaked credentials when contacting multiple > catalogs — #14178 <https://github.com/apache/iceberg/pull/14178> > - > > LICENSE/NOTICE: Update after adding iceberg-bigquery to runtime jars — > #14335 <https://github.com/apache/iceberg/pull/14335> > > Thanks, > Huaxin > > On Thu, Oct 16, 2025 at 9:33 PM Aihua Xu <[email protected]> wrote: > >> Hi Huaxin, >> >> Can we also include https://github.com/apache/iceberg/pull/14261 in >> 1.10.1 milestone which could cause issue in Spark against variant column? >> >> Thanks, >> Aihua >> >> >> >> On Sep 30, 2025, at 11:37 PM, Eduard Tudenhöfner < >> [email protected]> wrote: >> >> We might want to consider adding >> https://github.com/apache/iceberg/pull/14221 as well, since that fixes >> https://github.com/apache/iceberg/issues/14219. >> >> On Tue, Sep 30, 2025 at 6:45 PM huaxin gao <[email protected]> >> wrote: >> >>> Thanks, Amogh and Péter, for adding these PRs to the 1.10.1 milestone. >>> I’ve also added https://github.com/apache/iceberg/pull/14186 and >>> https://github.com/apache/iceberg/pull/13804 to the milestone. >>> >>> On Tue, Sep 30, 2025 at 8:38 AM Péter Váry <[email protected]> >>> wrote: >>> >>>> Thanks Hoaxin for taking care of this! >>>> >>>> I have added a Flink PR (and the backport of it) to he milestone, >>>> which could cause correctness issues with the Flink Sink: >>>> >>>> - https://github.com/apache/iceberg/pull/14182 >>>> - https://github.com/apache/iceberg/pull/14213 >>>> >>>> Thanks aiborodin for reporting, and Max for providing a quick fix >>>> until we have a better solution! >>>> >>>> Amogh Jahagirdar <[email protected]> ezt írta (időpont: 2025. szept. >>>> 30., K, 2:09): >>>> >>>>> Thanks Huaxin for volunteering! I also went ahead and added >>>>> https://github.com/apache/iceberg/pull/14161 to the 1.10.1 milestone >>>>> which also looks to address the S3FileIO close issue mentioned earlier in >>>>> the thread. >>>>> >>>>> On Thu, Sep 25, 2025 at 1:15 PM huaxin gao <[email protected]> >>>>> wrote: >>>>> >>>>>> Thanks for kicking this off, Amogh, and thanks everyone for the fix >>>>>> pointers. >>>>>> >>>>>> I’m happy to volunteer as release manager for 1.10.1. I’ll track the >>>>>> candidates mentioned in this thread, monitor review progress, and verify >>>>>> backports. >>>>>> >>>>>> If there are other must-have fixes, please reply here so we can >>>>>> consider them for the patch. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Huaxin >>>>>> >>>>>> On Tue, Sep 23, 2025 at 12:53 PM Amogh Jahagirdar <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Thanks all! >>>>>>> >>>>>>> >Would it be possible to include >>>>>>> https://github.com/apache/iceberg/pull/14153 to enable variant >>>>>>> shredding >>>>>>> >>>>>>> Regarding this one, I don't think it should be in a patch release >>>>>>> since it's exposing an API on the data file writer and not really a fix. >>>>>>> Besides it is technically possible already to just use the lower level >>>>>>> parquet writer with the variant shredding spec and then produce a data >>>>>>> file >>>>>>> from that (it's a bit more clunky for sure which is why that API change >>>>>>> in >>>>>>> the PR is helpful) but don't think it quite meets the criteria for >>>>>>> being in >>>>>>> a minor release. That criteria is primarily based on fixes. >>>>>>> >>>>>>> Thanks, >>>>>>> Amogh Jahagirdar >>>>>>> >>>>>>> >>>>>>> On Tue, Sep 23, 2025 at 10:13 AM Jean-Baptiste Onofré < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi Amogh >>>>>>>> >>>>>>>> Thanks for starting the discussion! >>>>>>>> >>>>>>>> I will create the PR to "relax" the LICENSE/NOTICE (to avoid the >>>>>>>> update pain before release). >>>>>>>> >>>>>>>> Regards >>>>>>>> JB >>>>>>>> >>>>>>>> On Mon, Sep 22, 2025 at 6:29 PM Amogh Jahagirdar <[email protected]> >>>>>>>> wrote: >>>>>>>> > >>>>>>>> > Hey folks, >>>>>>>> > >>>>>>>> > Iceberg 1.10 was released 2 weeks ago and there was one issue >>>>>>>> around incorrect variant filtering reported that I think meets the >>>>>>>> criteria >>>>>>>> for a patch release. The fix PR is in (thank you Drew). I wanted to >>>>>>>> kick >>>>>>>> this discussion thread off in case folks had other issues in the 1.10 >>>>>>>> release that they think warrant a patch release. >>>>>>>> > >>>>>>>> > I also think this PR is a good candidate for a patch release; >>>>>>>> this is for addressing a long-standing issue where closing the S3FileIO >>>>>>>> during an event like moving broadcast variables from memory to disk >>>>>>>> leads >>>>>>>> to an unexpected closing of the http client. There's still some >>>>>>>> discussion >>>>>>>> on the approach of the fix but there's general recognition that it's a >>>>>>>> legitimate issue, so I think it'd be ideal to get this in for a patch >>>>>>>> release as well. >>>>>>>> > >>>>>>>> > I've also created a milestone here. >>>>>>>> > >>>>>>>> > Thanks, >>>>>>>> > >>>>>>>> > Amogh Jahagirdar >>>>>>>> >>>>>>> >>
