morningman commented on issue #1757: kafka routine load kafka consume failed Broker: Offset out of range URL: https://github.com/apache/incubator-doris/issues/1757#issuecomment-529214784 > 1. routine load过程中没有动态的考虑offset均衡问题，每个BE始终分配的那几个固定的partition We will find a way to balance the partition > 2. 消费kafka中已经过期的offset对应的数据的时候应该跳过并继续往后消费，而不是报Offset out of range并暂停routine load I think it is not safe to just skip and continue. We add a new feature that kafka client will commit the consumed offset back to kafka, so that the kafka can delete the messages before the committed offset. You can see this PR: https://github.com/apache/incubator-doris/pull/1734
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
