Yes, I understand that is how it works today and I'll use a cron job or
something to create these as needed. I'll also look into the bucketing by
hour aspect.
That said, I'm suggesting that perhaps an alternative implementation (and
associated abstraction/plugability) may be of value.
On Mon,
whenever you create a partition in hive, it needs to be registered with the
metadata store. So short answer would be partition data is looked from
metadata store instead of the actual source data.
having a lot of partitions does slow down hive (around 1+). Normally
have not seen anyone using h
