Re: [DISCUSS] Apache Iceberg Release 1.3.1

[Szehon Ho]

Hi guys

Just an update on this.  Another issue came up about the new 1.3.0 function
rewrite_position_deletes (thanks Fokko for adding to the milestone).  I'm
working on that, hopefully can finish in next day or two, for this release.

Milestone for reference:
https://github.com/apache/iceberg/milestones/Iceberg%201.3.1

Thanks
Szehon

On Mon, Jul 10, 2023 at 11:14 AM Szehon Ho  wrote:

> Thanks Eduard!  Merged all your backport prs, I will commit the last one
> probably tomorrow and then we can start the release.
>
> Thanks
> Szehon
>
> On Sun, Jul 9, 2023 at 11:53 PM Eduard Tudenhoefner 
> wrote:
>
>> I created a 1.3.x  branch,
>> so that we can start backporting those bug fixes.
>>
>> Eduard
>>
>> On Fri, Jul 7, 2023 at 6:52 PM Szehon Ho  wrote:
>>
>>> Thanks a lot Eduard!  I think
>>> https://github.com/apache/iceberg/pull/7933 is also a good candidate as
>>> well.
>>>
>>> Thanks,
>>> Szehon
>>>
>>> On Fri, Jul 7, 2023 at 9:07 AM Eduard Tudenhoefner 
>>> wrote:
>>>
 +1 for a 1.3.1 release. I've created a 1.3.1 Milestone
 
 and it would be great to also get #7621
  in.

 Eduard

 On Fri, Jul 7, 2023 at 5:52 PM Ryan Blue  wrote:

> +1 for a 1.3.1 to fix the Hive issue.
>
> For the Nessie changes, those seem outside what we would normally put
> in a patch release. Patch releases are for bug fixes and aren't usually a
> time to get other changes in for convenience. I can understand wanting to
> unblock a Trino issue, but it doesn't seem like a good choice to me.
>
> In addition, why not put some of these classes in the Nessie project
> itself? Could NessieUtil go there so that you aren't waiting on Iceberg
> releases to fix third-party projects?
>
> Ryan
>
> On Thu, Jul 6, 2023 at 9:02 PM Jean-Baptiste Onofré 
> wrote:
>
>> Hi,
>>
>> It sounds good to me to have 1.3.1.
>>
>> Thanks !
>> Regards
>> JB
>>
>> On Fri, Jul 7, 2023 at 12:53 AM Szehon Ho 
>> wrote:
>> >
>> > Hi
>> >
>> > I wanted to start a discussion for whether its the right time for
>> 1.3.1, a patch release of 1.3.0.  It was started based on the issue found
>> by Xiangyang (@ConeyLiu) :
>> https://github.com/apache/iceberg/pull/7931#pullrequestreview-1507935277
>> .
>> >
>> > Do people have any other bug fixes that should be included?  Also
>> let me know, if anyone wants to be a release manager?  If not, I can give
>> it a shot as well.
>> >
>> > Thanks,
>> > Szehon
>>
>
>
> --
> Ryan Blue
> Tabular
>

Re: [DISCUSS] Apache Iceberg Release 1.3.1

[Fokko Driesprong]

Hi Szehon,

Thank you for the updates. I'm in favor of 1.3.1 as well. I got notified of
a discrepancy  in Java's
TableMetadata reader today. I have a fix here
 against the master branch.
Once that is in, I think it would be great to backport this to 1.3.x as
well.

Kind regards,
Fokko

Op wo 12 jul 2023 om 22:09 schreef Szehon Ho :

> Hi guys
>
> Just an update on this.  Another issue came up about the new 1.3.0
> function rewrite_position_deletes (thanks Fokko for adding to the
> milestone).  I'm working on that, hopefully can finish in next day or two,
> for this release.
>
> Milestone for reference:
> https://github.com/apache/iceberg/milestones/Iceberg%201.3.1
>
> Thanks
> Szehon
>
> On Mon, Jul 10, 2023 at 11:14 AM Szehon Ho 
> wrote:
>
>> Thanks Eduard!  Merged all your backport prs, I will commit the last one
>> probably tomorrow and then we can start the release.
>>
>> Thanks
>> Szehon
>>
>> On Sun, Jul 9, 2023 at 11:53 PM Eduard Tudenhoefner 
>> wrote:
>>
>>> I created a 1.3.x 
>>> branch, so that we can start backporting those bug fixes.
>>>
>>> Eduard
>>>
>>> On Fri, Jul 7, 2023 at 6:52 PM Szehon Ho 
>>> wrote:
>>>
 Thanks a lot Eduard!  I think
 https://github.com/apache/iceberg/pull/7933 is also a good candidate
 as well.

 Thanks,
 Szehon

 On Fri, Jul 7, 2023 at 9:07 AM Eduard Tudenhoefner 
 wrote:

> +1 for a 1.3.1 release. I've created a 1.3.1 Milestone
> 
> and it would be great to also get #7621
>  in.
>
> Eduard
>
> On Fri, Jul 7, 2023 at 5:52 PM Ryan Blue  wrote:
>
>> +1 for a 1.3.1 to fix the Hive issue.
>>
>> For the Nessie changes, those seem outside what we would normally put
>> in a patch release. Patch releases are for bug fixes and aren't usually a
>> time to get other changes in for convenience. I can understand wanting to
>> unblock a Trino issue, but it doesn't seem like a good choice to me.
>>
>> In addition, why not put some of these classes in the Nessie project
>> itself? Could NessieUtil go there so that you aren't waiting on Iceberg
>> releases to fix third-party projects?
>>
>> Ryan
>>
>> On Thu, Jul 6, 2023 at 9:02 PM Jean-Baptiste Onofré 
>> wrote:
>>
>>> Hi,
>>>
>>> It sounds good to me to have 1.3.1.
>>>
>>> Thanks !
>>> Regards
>>> JB
>>>
>>> On Fri, Jul 7, 2023 at 12:53 AM Szehon Ho 
>>> wrote:
>>> >
>>> > Hi
>>> >
>>> > I wanted to start a discussion for whether its the right time for
>>> 1.3.1, a patch release of 1.3.0.  It was started based on the issue 
>>> found
>>> by Xiangyang (@ConeyLiu) :
>>> https://github.com/apache/iceberg/pull/7931#pullrequestreview-1507935277
>>> .
>>> >
>>> > Do people have any other bug fixes that should be included?  Also
>>> let me know, if anyone wants to be a release manager?  If not, I can 
>>> give
>>> it a shot as well.
>>> >
>>> > Thanks,
>>> > Szehon
>>>
>>
>>
>> --
>> Ryan Blue
>> Tabular
>>
>

Re: Code review: [spark] skip empty file during table migration, table snapshotting or adding files

[Pucheng Yang]

Hi community,

I raised another PR on the same topic, can you help review? Thanks

PR1: https://github.com/apache/iceberg/pull/8040 [spark 3.4] skip empty
file during table migration, table snapshotting or adding files
PR2: https://github.com/apache/iceberg/pull/8051 [spark 3.4] add
remove_empty_files procedure

On Tue, Jul 11, 2023 at 10:07 AM Pucheng Yang  wrote:

> Hi community,
>
> In a previous email, I asked about how to get rid of partitions that only
> contain empty files.
>
> Here I am proposing a PR https://github.com/apache/iceberg/pull/8040
> (issue: https://github.com/apache/iceberg/issues/7949) to skip adding
> empty files during the migration, snapshotting or adding files such that we
> can prevent issue like above happens.
>
> Could you please take a look? Thanks
>
> Best,
> Pucheng
>