I am using Glue catalog. I figured out the issue here. When i used rowDelta
transaction and FanoutWriters the issue was resolved and data was available
in Athena.
Thanks for the input Ryan
On Thu, 30 Jun, 2022, 4:29 am Ryan Blue, wrote:
> Taher, what catalog are you using? You would need to use
Is "truncate" not an option? This would do a table wide delete which would
create a new snapshot which you can keep. No data files would be valid
after this?
On Wed, Jun 29, 2022 at 6:29 PM Steve Zhang
wrote:
> Hey Iceberg Community:
>
> I am wondering if there’s any best practice to handle resi
Hey Iceberg Community:
I am wondering if there’s any best practice to handle residual of data files
deleted from last snapshot in the iceberg table.
Let me explain the use case here, considering the data retention policy in
place where some of the sensitive data can only be stored on disk for
Taher,
I typically use the helpers in the `Parquet` class to create Parquet files.
That's probably the easiest way to create individual files.
`FileWriterFactory` and `FileAppenderFactory` are ways to provide object
model support to common write patterns. Flink and Spark both use different
in-mem
Taher, what catalog are you using? You would need to use the same catalog
that Athena is talking to.
On Mon, Jun 27, 2022 at 10:55 PM Taher Koitawala wrote:
> Hi All,
> I am trying to append data files to an existing iceberg table in
> Athena. I wrote a Java app that uses the Iceberg a
Hi Vinod,
The problem of moving an Iceberg table is the absolute path within the
metadata layer. They may not work once you move it from one place to
another. Here are solutions:
1. Make sure the paths are the same, or it works across regions. For
example, you can use
https://docs.aws.amazon.com/A
