Re: [DISCUSS] PIP-215: Configurable Topic Compaction Strategy

2022-10-29 Thread Heesung Sohn
Hi, Also, I thought about some concurrent assignment scenarios between pre-filter vs post-filter. Example 1: When the topic broadcast is slower than the concurrent assignment requests With pre-filter + two-topics (non-compacted and compacted topics) t1: A -> non-compacted topic // broker A publi

Re: [DISCUSS] PIP-215: Configurable Topic Compaction Strategy

2022-10-29 Thread Heesung Sohn
Hi Michael, For the pre-prefilter(pre-compaction) option, I think the leader requires a write-through cache to compact messages based on the latest states. Otherwise, the leader needs to wait for the last msg from the (compacted) topic before compacting the next msg for the same bundle. Pulsar gu