I think the rough calculation of max memory footprint for each high level consumer would be:
(Number Of Partitions For All Topics) * fetch.message.max.bytes * queued.max.message.chunks + (some decompression memory cost for a message) In your case, it would be 10 times above. Thanks, Jiangjie (Becket) Qin On 7/16/15, 1:40 PM, "Kris K" <squareksc...@gmail.com> wrote: >Hi All, > >Is there a way to calculate the amount of memory used per thread in case >of >a high level consumer? > >I am particularly interested in calculating the memory required by a >process running 10 high level consumer threads for 15 topics with max. >file >size set to 100 MB. > >Thanks, >Kris
