Thanks for the KIP, +1 from me. I have a few comments with regards to the method names chosen, but since none of the classes in question are public API, I'll comment directly in the PR.
Ismael On Mon, Nov 7, 2016 at 9:08 PM, radai <radai.rosenbl...@gmail.com> wrote: > Hi, > > I would like to initiate a vote on KIP-72: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 72%3A+Allow+putting+a+bound+on+memory+consumed+by+Incoming+requests > > The kip allows specifying a limit on the amount of memory allocated for > reading incoming requests into. This is useful for "sizing" a broker and > avoiding OOMEs under heavy load (as actually happens occasionally at > linkedin). > > I believe I've addressed most (all?) concerns brought up during the > discussion. > > To the best of my understanding this vote is about the goal and > public-facing changes related to the new proposed behavior, but as for > implementation, i have the code up here: > > https://github.com/radai-rosenblatt/kafka/tree/broker- > memory-pool-with-muting > > and I've stress-tested it to work properly (meaning it chugs along and > throttles under loads that would DOS 10.0.1.0 code). > > I also believe that the primitives and "pattern"s introduced in this KIP > (namely the notion of a buffer pool and retrieving from / releasing to said > pool instead of allocating memory) are generally useful beyond the scope of > this KIP for both performance issues (allocating lots of short-lived large > buffers is a performance bottleneck) and other areas where memory limits > are a problem (KIP-81) > > Thank you, > > Radai. >
