Hi all, I have a Paimon table that is continuously populated by a streaming job based on incoming data. I need to evict records from the table once they are no longer used—where "not used" can be defined by a SQL query.
>From what I’ve observed, running such a query in a separate batch job requires suspending the streaming job. This is because multiple writers to the same Paimon table can cause conflicts, potentially resulting in job failures. Additionally, I haven't found a way to periodically schedule such a cleanup query within the streaming application itself. What would be the best way to handle this use case?
