Shekharrajak commented on issue #18439: URL: https://github.com/apache/druid/issues/18439#issuecomment-4289388215
After working on couple of benchmarks, I realised there is overahead in kafka broker side for share group due to - All consumers **share the same partition** concurrently - Broker manages per-batch state tracking, locking, and acknowledgment - Prevents double-delivery via batch-level locks - Serializes access when multiple consumers compete for the same partition but still get better results because we can scale number of tasks using lag metrics. Traditional Kafka is doing good if records are evenly distributed in partitions, throughput and overall latency. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
