Sure, you could implement that in a catalog, either one that you plug in or contribute it to the Iceberg HiveCatalog.
On Mon, Feb 1, 2021 at 3:01 AM luochong.lxf <luochong....@gmail.com> wrote: > Hi Ryan, > > Generally we add createTime and modifiedTime in table schema. However, due > to historical reasons some hive tables do not have createTime and > modifiedTime. When these hive tables are transformed to iceberg tables, we > hope createTime and transient_lastDdl can be retained, so we can still do > data expiration and track table activities. Once snapshots expire, we can > not get these time info from iceberg. It seems the only way to solve the > problem I mentioned above is to modify these hive table schema and rewrite > them. Do you think so? > > Thanks > > luochong.lxf > luochong....@gmail.com > > <https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1&name=luochong.lxf&uid=luochong.lxf%40gmail.com&iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fqiyelogo%2FdefaultAvatar.png&items=%5B%22luochong.lxf%40gmail.com%22%5D> > 签名由 网易邮箱大师 <https://mail.163.com/dashi/dlpro.html?from=mail81> 定制 > > On 01/29/2021 02:40，Ryan Blue<rb...@netflix.com.INVALID> > <rb...@netflix.com.INVALID> wrote： > > Chong, > > Once snapshots expire, I don't think that there is a way to recover the > time that a given partition was created. > > Can you explain more about what you're trying to do? When we age off data, > we use the age of the records themselves, not the age from metadata. In > other words, we use the logical timestamp from a row to expire it, not the > timestamp when it was added to the table. You might consider doing that as > well. I think it is probably a better way to ensure compliance. > > rb > > On Thu, Jan 28, 2021 at 9:42 AM chong luo <luochong....@gmail.com> wrote: > >> Hi Iceberg Devs >> >> >> I’m currently working on delete expired table and partition in iceberg. >> However, I can not find table/partition creation time, it seems iceberg >> only stores snapshot creation time. In hive, transient_lastDdlTime, >> createTime and lastAccessTime are stored in metastore. With time metadata, >> we can know when table is changed and track related jobs. >> >> >> Is there any way to get the time metadata mentioned above in the current >> implementation of iceberg? >> >> >> >> Thanks. >> > > > -- > Ryan Blue > Software Engineer > Netflix > > -- Ryan Blue Software Engineer Netflix
