Re: [EXTERNAL] Partial data with ADLS Gen2

2022-07-26 Thread hwl17801341688
D Replied Message | From | Tufan Rakshit | | Date | 07/24/2022 18:59 | | To | Shay Elbaz | | Cc | kineret M, user | | Subject | Re: [EXTERNAL] Partial data with ADLS Gen2 | Just use Delta Best Tufan Sent from my iPhone On 24 Jul 2022, at 12:20, Shay Elbaz wrote:  This is a

Re: [EXTERNAL] Partial data with ADLS Gen2

2022-07-24 Thread Tufan Rakshit
Just use Delta Best Tufan Sent from my iPhone > On 24 Jul 2022, at 12:20, Shay Elbaz wrote: > >  > This is a known issue. Apache Iceberg, Hudi and Delta lake and among the > possible solutions. > Alternatively, instead of writing the output directly to the "official" > location, write it t

Re: [EXTERNAL] Partial data with ADLS Gen2

2022-07-24 Thread Shay Elbaz
This is a known issue. Apache Iceberg, Hudi and Delta lake and among the possible solutions. Alternatively, instead of writing the output directly to the "official" location, write it to some staging directory instead. Once the job is done, rename the staging dir to the official location. __