Re: Doubts

2019-07-17 Thread Omar Al-Safi
ii) If you want to move data in-house that in a DB (let's say a rational DB like MySql ..etc), I strongly advise you to look at https://debezium.io/ which is a CDC (Change Data Capture) Kafka Connect plugin which records DDL events from your DB and propagate them directly to Kafka, if you have a Ka

Re: Best practices for compacting topics with tombstones

2019-07-18 Thread Omar Al-Safi
If I recall correctly, you can set 'delete.retention.ms' in the topic level configuration to how long you want to retain the tombstones in the topic. By default is set to 8640, you can set it to lower than this. Regarding the performance, I am not really why would compaction causes the performa