viktorsomogyi commented on PR #15697: URL: https://github.com/apache/kafka/pull/15697#issuecomment-2098821932
@soarez at the end I chose the shortcut regarding detecting leaders before shutdown. The reason is complex as the solution that would be required for this is complex too. So on one part the sequence of events is problematic. First we update the `LogManager` and then try to propagate the event to the controller. At this point the metadata is stale so I can't use that for reliable information to detect whether partitions have leadership or not. A workaround would be to subtract the LogManager's data from metadata cache (ie. if there is only a single isr replica and that is the current, then we can accept it as offline in reality). I don't really feel that it is a robust solution, it could be prone to race conditions on the network depending on how requests come from the controller as long as it's alive. I think it's more robust to just fail if we can't contact the controller. The second reason is a bit technical and can be worked around, although requires lots of effort. When trying to extract which replica->logdir information from `LogManager`, my only available information regarding logdirs given by the event is the `Uuid`. Unfortunately `LogManager` doesn't store the `Uuid` of an offline dir (and besides I don't think `Uuid` and logdir names used consistently across the whole module). This problem can be solved by propagating both logdir and `Uuid` in the events or store offline dirs' `Uuid ` in `LogManager`. I think the latter is problematic because we can't know the point until we should store information about offline dirs as they might never come back. The first can be done, although could be a sizeable refactor and generally I felt that just choosing the simpler route now could be more robust. Let me know if you think we should try it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
