Re: How to add Datafiles from an existing iceberg table?

2019-12-23 Thread Ryan Blue
Hi Junjie, The problem is that your writer doesn't have the same schema as the records you're passing to it, because addedFiles doesn't project all columns. Writers assume that the write schema and record schema matches, and will throw an exception like this if they don't. The projection in added

How to add Datafiles from an existing iceberg table?

2019-12-23 Thread Junjie Chen
Hi community I tried to add data files from an existing iceberg table to a target iceberg table with following code (unit test): Iterator datafiles = sourceTable.currentSnapshot().addedFiles().iterator(); while (datafiles.hasNext()) { targetTable.newAppend().appendFile(datafiles.ne