Hi Lawrence, > As I'm thinking through this a little more, if that is the case and the node is removed, some partitions in the system may be marked as under-replicated and cause a cascading effect where partitions are re-replicated and cause other nodes to fill up. Has that ever happened? Does Kafka have a contingency plan for such a scenario?
Currently, Kafka doesn't rebalance partitions automagically if there is an issue with a broker. That excludes the failure scenario that you portrait. Regarding how Kafka handles a full disk, I can't answer that. Cheers, Jens On Wed, Apr 20, 2016 at 7:09 PM Lawrence Weikum <lwei...@pandora.com> wrote: > Hello, > > I'm curious about the expected or default behavior that might occur if a > broker in the system has filled up. By that I mean when a broker has used > all of its memory and disk space. Is the node simply removed from the > system until space is cleared? > > As I'm thinking through this a little more, if that is the case and the > node is removed, some partitions in the system may be marked as > under-replicated and cause a cascading effect where partitions are > re-replicated and cause other nodes to fill up. Has that ever happened? > Does Kafka have a contingency plan for such a scenario? > > Thank you so much for your insight and all of your hard work! > > Lawrence > -- Jens Rantil Backend Developer @ Tink Tink AB, Wallingatan 5, 111 60 Stockholm, Sweden For urgent matters you can reach me at +46-708-84 18 32.
