I have a question on how hidden partitioning works in Iceberg using Java
API.
The code is something like the following.

```
// records is the list of records with a time column
// table is created using partition spec hour(time)
// records have different rows with different hours

Table table = loadTable();

Path path = new Path(...);
FileAppender<Record> appender = Avro.write(fromPath(path, conf)).build();
appender.addAll(records);
appender.close();

DataFile dataFile = DataFiles.builder(table.spec())

 .withInputFile(HadoopInputFile.fromPath(path, conf))
                                 .build();

table.newAppend().appendFile(dataFile).commit();
```
However, once committed, I still see only one partition count updated and
one data file persisted, even though the underlying records
spread different hours.

I think I use the API in the wrong way but appreciate if someone can help
me on the right way to write partitioned data.


Thanks,
-- 
Chen Song

Reply via email to