bq. only one specific node is showing this issue Is controller running on this node ? updating the metrics is expensive.
Cheers On Sat, Aug 4, 2018 at 3:00 PM Abhijith Sreenivasan < abhijithonl...@gmail.com> wrote: > Hello > > We are seeing high CPU usage for the Kafka process. I am using 0.11 > version. Has 5 topics out of which 1 was created newly. We attempted to > publish message this new topic which did not show up in the consumer, but > no errors in the publisher end. Not sure why the message did not show up in > consumer. > > This ran for a couple of days (30K messages) when we noticed 100%+ CPU > usage. Tried deleting the topic (config is enabled), it was marked for > deletion but after which usage rose to below levels 240%+. We restarted the > process many times and disabled the publisher/producer but no difference. > After some time (1 or 2 hours) we are getting a "Too many open files" error > and process is shutting down. > > We have 3 nodes with Kafka and 3 other nodes running with ZK, but only one > specific node is showing this issue. (where new topic partition is > present). > > Still debugging and this is a prod environment.. please help! > > Thanks, > Abhi > > top - 17:47:43 up 289 days, 18:54, 2 users, load average: 2.65, 2.72, > 2.52 > Tasks: 144 total, 1 running, 143 sleeping, 0 stopped, 0 zombie > %Cpu(s): 37.5 us, 19.4 sy, 0.0 ni, 37.0 id, 0.0 wa, 0.0 hi, 5.4 si, > 0.7 st > KiB Mem : 16266464 total, 1431916 free, 5769976 used, 9064572 buff/cache > KiB Swap: 0 total, 0 free, 0 used. 9230548 avail Mem > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 32058 root 20 0 5898348 1.078g 15548 S 253.0 6.9 99:03.68 java > 10 root 20 0 0 0 0 S 0.3 0.0 921:42.77 > rcu_sche >
