Re: Apache Kafka Process showing high CPU (100 to 200+) usage in Linux when Idle

2018-08-04 Thread Manjunath N
After you deleted a topic was it a clean delete. Did you verify in zookeeper and kafka logs directory? if not you may need to do some clean up if there are inconsistency in kafka logs dir and zookeeper. did you try to move the replicas assignment to different machines for this topic and see if i

Re: Viewing transactional markers in client

2018-08-04 Thread Cody Koeninger
Here's one reason I might want to be able to tell whether a given offset is a transactional marker: https://issues.apache.org/jira/browse/SPARK-24720 Alternatively, is there any efficient way to tell what the offset of the last actual record in a topicpartition is (i.e. like endOffsets) On Thu,

Re: Apache Kafka Process showing high CPU (100 to 200+) usage in Linux when Idle

2018-08-04 Thread Ted Yu
bq. only one specific node is showing this issue Is controller running on this node ? updating the metrics is expensive. Cheers On Sat, Aug 4, 2018 at 3:00 PM Abhijith Sreenivasan < abhijithonl...@gmail.com> wrote: > Hello > > We are seeing high CPU usage for the Kafka process. I am using 0.11

Apache Kafka Process showing high CPU (100 to 200+) usage in Linux when Idle

2018-08-04 Thread Abhijith Sreenivasan
Hello We are seeing high CPU usage for the Kafka process. I am using 0.11 version. Has 5 topics out of which 1 was created newly. We attempted to publish message this new topic which did not show up in the consumer, but no errors in the publisher end. Not sure why the message did not show up in co

Re: Visualizing kafka topics data

2018-08-04 Thread Jonathan Santilli
Hello Akash, we use for simple cases Kafdrop: https://github.com/HomeAdvisor/Kafdrop Is a jar that raises an HTTP server. Hope it helps. -- Jonathan On Sat, Aug 4, 2018 at 12:56 AM Rahul Singh wrote: > Graylog, or Kibana (connected to ElasticSearch) > > > Rahul > On Aug 3, 2018, 5:12 PM -04