Taher, what catalog are you using? You would need to use the same catalog
that Athena is talking to.
On Mon, Jun 27, 2022 at 10:55 PM Taher Koitawala <taher...@gmail.com> wrote:
> Hi All,
> I am trying to append data files to an existing iceberg table in
> Athena. I wrote a Java app that uses the Iceberg apis to do this. Attached
> below is a sample source code. I do see parquet files being placed on s3
> however I cannot read those files in Athena. Am I missing something?
>
> ```
> //Use glueCatalog and loaded table
> Table table =
> catalog.loadTable(TableIdentifier.of(Namespace.of("tpc_ds"),"store_sales"));
>
> //created a file appender
> FileAppender<Record> appender
> =factory.newAppender(S3OutputFile.fromLocation(path,s3Client),
> FileFormat.PARQUET);
>
> //Added records to appender
> appender.add(record);
>
> //get len and close appender
> long fileLen = appender.length();
> appender.close();
>
> //get metrics
> appender.metrics();
>
> //register datafiles with table
> Transaction transaction = table.newTransaction();
>
> //Append data file
> AppendFiles appendFiles = transaction.newAppend();
> appendFiles.appendFile(DataFiles.builder(spec)
> .withMetrics(metrics)
> .withPath(location)
> .withFormat(FileFormat.PARQUET)
> .withRecordCount(metrics.recordCount())
> .withFileSizeInBytes(fileLen).
> build());
>
> //commit append
> appendFiles.commit();
>
> //commit transaction
> transaction.commitTransaction();
>
> --program end
>
> ```
>
> Regards,
> Taher Koitawala
>
--
Ryan Blue
Tabular
