Hi community, In my organization, a big portion of the datasets are partitioned by date, normally we keep the latest X dates of partition for a given dataset.
One issue that always bothers me is if I want to delete a partition that should be GC, I will run SQL query "delete from tbl where dt = ..." and do snapshot expiration to keep the latest snapshot to make sure that partition data is physically removed. However, the downside of this approach is the table snapshot history will be completely lost.. I wonder if anyone else in the community has the same pain point? How do you solve this? I would love to understand if there is a solution to this otherwise we can brainstorm if there is a way to solve this. Thanks! Pucheng
