Delta now exposes this functionality as a command, and some groups (like ours) have some internal functionality for doing this. I think it's worth reconsidering this as a first class procedure in the Iceberg-Spark module since we get a lot of requests about it and now position deletes are a bit more complicated.
On Sat, Dec 2, 2023 at 5:35 PM Wing Yew Poon <wyp...@cloudera.com.invalid> wrote: > Aren't we forgetting about position delete files? If the table has > position delete files, then those contain absolute file paths as well. > We cannot add them to the table as-is. We need to rewrite them. This, I > think, is the most painful part of replicating an Iceberg table. > - Wing Yew > > > On Sat, Dec 2, 2023 at 5:23 PM Fokko Driesprong <fo...@apache.org> wrote: > >> Hi Dongjun, >> >> Thanks for reaching out on the mailinglist. Another option might be to >> copy the data, and then use a Spark procedure, called add_files >> <https://iceberg.apache.org/docs/latest/spark-procedures/#add_files> to >> add the files to the table. Let me know if this works for you. >> >> Kind regards, >> Fokko >> >> Op za 2 dec 2023 om 02:43 schreef Ajantha Bhat <ajanthab...@gmail.com>: >> >>> Hi, >>> >>> You are right. Moving Iceberg tables from storage and expecting them to >>> function at the new location is not currently feasible. >>> The issue lies in the metadata files, which store the absolute path. >>> >>> To address this, we need support for relative paths, but it appears that >>> progress on this front has been slow. >>> You can monitor the status of this feature at >>> https://github.com/apache/iceberg/pull/8260. >>> >>> As a temporary fix, you can use the CTAS method to create a duplicate >>> copy of the table at the desired new path. >>> >>> Thanks, >>> Ajantha >>> >>> On Fri, Dec 1, 2023 at 10:01 PM Dongjun Hwang <enter09...@gmail.com> >>> wrote: >>> >>>> Hello! My name is Dongjun Hwang. >>>> >>>> I recently performed distcp on the iceberg table in Hadoop. >>>> >>>> Data search was not possible because all file paths in the metadata >>>> directory were not changed. >>>> >>>> Is there a way to distcp the iceberg table? >>>> >>>> thang you!! >>>> >>>
