Re: Best practices for compacting topics with tombstones

2019-07-18 Thread Omar Al-Safi
If I recall correctly, you can set 'delete.retention.ms' in the topic level configuration to how long you want to retain the tombstones in the topic. By default is set to 8640, you can set it to lower than this. Regarding the performance, I am not really why would compaction causes the performa

Best practices for compacting topics with tombstones

2019-07-17 Thread Chris Baumgartner
Hello, I'm wondering if anyone has advice on configuring compaction. Here is my scenario: A producer writes raw data to topic #1. A stream app reads the data from topic #1, processes it, writes the processed data to topic #2, and then writes a tombstone record to topic #1. So, I don't intend for