Hi, I would like to initiate a vote on KIP-72:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-72%3A+Allow+putting+a+bound+on+memory+consumed+by+Incoming+requests The kip allows specifying a limit on the amount of memory allocated for reading incoming requests into. This is useful for "sizing" a broker and avoiding OOMEs under heavy load (as actually happens occasionally at linkedin). I believe I've addressed most (all?) concerns brought up during the discussion. To the best of my understanding this vote is about the goal and public-facing changes related to the new proposed behavior, but as for implementation, i have the code up here: https://github.com/radai-rosenblatt/kafka/tree/broker-memory-pool-with-muting and I've stress-tested it to work properly (meaning it chugs along and throttles under loads that would DOS 10.0.1.0 code). I also believe that the primitives and "pattern"s introduced in this KIP (namely the notion of a buffer pool and retrieving from / releasing to said pool instead of allocating memory) are generally useful beyond the scope of this KIP for both performance issues (allocating lots of short-lived large buffers is a performance bottleneck) and other areas where memory limits are a problem (KIP-81) Thank you, Radai.
