Re: Regarding ingestion rate, topic size and retention period

2023-11-09 Thread Richard Bosch
Hi, You mention topic size, but I will explain using partition size. If you want to control the maximum size of all partitions combined, you need to take that value, divide it by the number of partitions and set the retention.bytes setting to the result. A topic partition is stored in segments, w

Regarding ingestion rate, topic size and retention period

2023-11-09 Thread Yeikel Santana
Hi all, This might be a common question, but unfortunately, I couldn't find a reliable answer or documentation to guide me. There are various conflicting ideas. If a producer tries to ingest at a faster rate than the configuration set in the topic, what will happen? Example: - Topic size: 10