This rewrite tool(https://github.com/apache/iceberg/pull/4705) works for COW. And it is not hard to extend it to support MOR. It needs blessing from the community if we want to merge it.
Yufei On Mon, Dec 4, 2023 at 8:05 AM Russell Spitzer <russell.spit...@gmail.com> wrote: > Delta now exposes this functionality as a command, and some groups (like > ours) have some internal functionality for doing this. I think it's worth > reconsidering this as a first class procedure in the Iceberg-Spark module > since we get a lot of requests about it and now position deletes are a bit > more complicated. > > On Sat, Dec 2, 2023 at 5:35 PM Wing Yew Poon <wyp...@cloudera.com.invalid> > wrote: > >> Aren't we forgetting about position delete files? If the table has >> position delete files, then those contain absolute file paths as well. >> We cannot add them to the table as-is. We need to rewrite them. This, I >> think, is the most painful part of replicating an Iceberg table. >> - Wing Yew >> >> >> On Sat, Dec 2, 2023 at 5:23 PM Fokko Driesprong <fo...@apache.org> wrote: >> >>> Hi Dongjun, >>> >>> Thanks for reaching out on the mailinglist. Another option might be to >>> copy the data, and then use a Spark procedure, called add_files >>> <https://iceberg.apache.org/docs/latest/spark-procedures/#add_files> to >>> add the files to the table. Let me know if this works for you. >>> >>> Kind regards, >>> Fokko >>> >>> Op za 2 dec 2023 om 02:43 schreef Ajantha Bhat <ajanthab...@gmail.com>: >>> >>>> Hi, >>>> >>>> You are right. Moving Iceberg tables from storage and expecting them to >>>> function at the new location is not currently feasible. >>>> The issue lies in the metadata files, which store the absolute path. >>>> >>>> To address this, we need support for relative paths, but it appears >>>> that progress on this front has been slow. >>>> You can monitor the status of this feature at >>>> https://github.com/apache/iceberg/pull/8260. >>>> >>>> As a temporary fix, you can use the CTAS method to create a duplicate >>>> copy of the table at the desired new path. >>>> >>>> Thanks, >>>> Ajantha >>>> >>>> On Fri, Dec 1, 2023 at 10:01 PM Dongjun Hwang <enter09...@gmail.com> >>>> wrote: >>>> >>>>> Hello! My name is Dongjun Hwang. >>>>> >>>>> I recently performed distcp on the iceberg table in Hadoop. >>>>> >>>>> Data search was not possible because all file paths in the metadata >>>>> directory were not changed. >>>>> >>>>> Is there a way to distcp the iceberg table? >>>>> >>>>> thang you!! >>>>> >>>>
