Re: How to get table/partition creation time/update time in iceberg

2021-02-01 Thread Ryan Blue
Sure, you could implement that in a catalog, either one that you plug in or contribute it to the Iceberg HiveCatalog. On Mon, Feb 1, 2021 at 3:01 AM luochong.lxf wrote: > Hi Ryan, > > Generally we add createTime and modifiedTime in table schema. However, due > to historical reasons some hive tab

Re: How to get table/partition creation time/update time in iceberg

2021-02-01 Thread luochong . lxf
Hi Ryan,Generally we add createTime and modifiedTime in table schema. However, due to historical reasons some hive tables do not have createTime and modifiedTime. When these hive tables are transformed to iceberg tables, we hope createTime and transient_lastDdl can be retained, s

Re: How to get table/partition creation time/update time in iceberg

2021-01-28 Thread Ryan Blue
Chong, Once snapshots expire, I don't think that there is a way to recover the time that a given partition was created. Can you explain more about what you're trying to do? When we age off data, we use the age of the records themselves, not the age from metadata. In other words, we use the logica

How to get table/partition creation time/update time in iceberg

2021-01-28 Thread chong luo
Hi Iceberg Devs I’m currently working on delete expired table and partition in iceberg. However, I can not find table/partition creation time, it seems iceberg only stores snapshot creation time. In hive, transient_lastDdlTime, createTime and lastAccessTime are stored in metastore. With time m

How to get table/partition creation time/update time in iceberg

2021-01-27 Thread luochong . lxf
Hi Iceberg DevsI’m currently working on delete expired table and partition in iceberg. However, I can not find table/partition creation time, it seems iceberg only stores snapshot creation time. In hive, transient_lastDdlTime, cre