Re: complicated logic for tombstone records

2020-01-08 Thread Jan Bols
Hi Boyang, Hi Alex, thank you for your reply. I can't use windowing so currently I'm managing removals by wrapping messages in a delete-aware wrapper whenever I have to do aggregation but this has a big impact on all the logic. For me the ideal situation would be to get a handle on the state stor

Re: complicated logic for tombstone records

2020-01-02 Thread Alex Brekken
Hi Jan, unfortunately there is no easy or automatic way to do this. Publishing null values directly to the changelog topics will remove them from the topic, but it won't remove the corresponding row from the RocksDB state store. (though deleting data programmatically from a state-store WILL also r

Re: complicated logic for tombstone records

2020-01-02 Thread Boyang Chen
Hey Jan, although I believe your case is much more complicated, but would time based retention work for you at all? If yes, time window store is like the best option. If no, streams has no out-of-box solution for invalidating the aggregation record. It seems at least we could provide an API to in