Re: Inconsistency with Zookeeper

Please check and ensure that each machine can recognize each other. It might be the case zookeepers and kafka are running seperately on each machine, but together they are not part of the same cluster. On 08-Aug-2015 6:34 am, "Scott Clasen" wrote: > each zk needs a myid file in the data dir, wit

Re: How can I unsubscribe from this mailing list

Still I am not able to unsubscribe and still getting mails. Please help me out as It is filling my inbox. On Fri, Jul 3, 2015 at 11:57 AM, Monika Garg wrote: > Thanks. > > I will try these to unsubscribe. > > On Thu, Jun 25, 2015 at 12:59 PM, Stevo Slavić wrote: > >&g

Re: How can I unsubscribe from this mailing list

Slavic. > > On Thu, Jun 25, 2015 at 9:25 AM, Monika Garg wrote: > > > Hi > > > > I want to unsubscribe from this mailing list. > > > > I have sent the unsubscription mail on below two ids : > > > > kafka-users-unsubscr...@incubator.apache.org > &g

How can I unsubscribe from this mailing list

Hi I want to unsubscribe from this mailing list. I have sent the unsubscription mail on below two ids : kafka-users-unsubscr...@incubator.apache.org unsubscr...@kafka.apache.org But delivery to these two addresses getting failed. Please help me how can I subscribe from this mailing list. --

Getting logs in kakfaServer-gc.log even if Log level is set to OFF in log4j.properties

Hi I am using kafka_2.8.0-0.8.0.tar.gz bundle. I have set Log level as "OFF" in log4j.properties file and according to this log level,the logging is off(there should be no log generation in any .log file). But Kafka is generating logs in kafkaServer-gc.log file.Please tell if logging of this par

Re: Kafka request.log size is increasing continuously

ore > messages. Even fetch request will only block on the broker for up to > max.wait.time. Also, the producer periodically issues metadata requests, > which are also logged in kafka-request.log. > > Thanks, > > Jun > > > On Thu, Jan 23, 2014 at 2:20 AM, Monika Garg wrote

Kafka request.log size is increasing continuously

Hi I have created one node kafka cluster and one topic name test1. Then I started ConsoleProducer and ConsoleConsumer.Both started successfully.But what I see is in the /kafka/kafka_2.8.0-0.8.0/logs/ directory -rw-rw-r-- 1 admin admin5203 Jan 23 15:37 controller.log *-rw-rw-r-- 1 admin admin

Re: Property Name Changes in Different Versions of Kafka

Thanks a lot Joe...all my doubts/questions are cleared. Thanks again...:)

Re: Property Name Changes in Different Versions of Kafka

itter.com/allthingshadoop> > / > > > On Wed, Jan 8, 2014 at 5:18 AM, Monika Garg wrote: > > > Hi, > > > > I have two doubts for changes in different versions of Kafka available: > > > > *Doubt1:* > >

Re: Property Name Changes in Different Versions of Kafka

might be the reason for change of log.dir to log.dirs). *Doubt2:* /bin/kafka-server-start.sh file export JMX_PORT till version kafka-0.8.0-beta1-src.tgz,but in stable latest version it is not exporting JMX_PORT.So why this has been done. On Wed, Jan 8, 2014 at 3:48 PM, Monika Garg wrote: >

Property Name Changes in Different Versions of Kafka

Hi, I have two doubts for changes in different versions of Kafka available: *Doubt1:* I am using Kafka from the version kafka-0.7.1-incubating-src.tgz. The latest stable version is kafka_2.8.0-0.8.0.tar.gz. But there is propertyName difference in config/server.properties file of different versi

Re: Kafka List Topic Command O/P and Jconsole o/p differs

wrong. On Thu, Nov 21, 2013 at 3:25 AM, Joel Koshy wrote: > Did you send any messages to those topics? I think we register those > mbeans only after messages have been received. > > On Wed, Nov 20, 2013 at 03:20:25PM +0530, Monika Garg wrote: > > Hi, > > I am g

Re: List of topics with JMX?

ics that > > have been received by the broker since the broker process started. If I > > restart the broker process, the counts all reset to 0 and then start > > incrementing again. > > > > > > On Nov 19, 2013, at 12:56 PM, Monika Garg wrote: > >

Kafka List Topic Command O/P and Jconsole o/p differs

Hi, I am getting a very surprising case : I am having a 2 nodes Kafka cluster and created some topics. When I am running the command : *bin/kafka-list-topic.sh --zookeeper 192.168.145.119:2181 *, I am getting the below output: topic: sample-tickpartition: 0

Re: Leader = -1 is coming when broker goes down

Hi Neha, In context of the same mail thread that you have already answered,I just want to clear one more doubt: If I am making a cluster with two brokers and a topic with 3 partitions and two replicas,then can leader = -1 come here in any case? On Tue, Oct 22, 2013 at 10:55 PM, Monika Garg

Re: ActiveControllerCount is always will be either 0 or 1 in 3 nodes kafka cluster?

are live, which points to a bug. > > -Neha > > > On Tue, Nov 19, 2013 at 9:53 AM, Monika Garg wrote: > > > Thanks Neha,Now got your point completely. > > > > One more small thing that I want to understand: > > > > The case when mbean values for Active

Re: ActiveControllerCount is always will be either 0 or 1 in 3 nodes kafka cluster?

oolean as well, we found the addition rule to be > easier. > > Thanks, > Neha > On Nov 19, 2013 1:48 AM, "Monika Garg" wrote: > > > Thanks for replying Neha. > > Yes I got the same thing as u written:There should only be one controller > > at any point of t

Re: List of topics with JMX?

kafka.server":type="BrokerTopicMetrics",name="AllTopicsMessagesInPerSec" > count > > be easier? > > Also, correct me if I am wrong, but I believe that these count values are > the total number of messages seen for a topic (or all topics) since the > Broker was started, not

Re: ActiveControllerCount is always will be either 0 or 1 in 3 nodes kafka cluster?

her broker. In general, you want to alert if the addition of the mbean > values for ActiveControllerCount across all brokers in a cluster != 1. > > Thanks, > Neha > > > On Mon, Nov 18, 2013 at 11:16 PM, Monika Garg > wrote: > > > Hi, > > > > there is

ActiveControllerCount is always will be either 0 or 1 in 3 nodes kafka cluster?

Hi, there is one Mbean in jconsole with name "kafka.controller":type="KafkaController",name="ActiveControllerCount", the Value of this Mbean is always 1 on one broker and 0 on all others remaining broker of the 3 nodes kafka cluster.Will it always be like this? As per my understanding it will al

Re: List of topics with JMX?

Yes,in jconsole the Mbean "kafka.server":type="BrokerTopicMetrics",contains all the details for all the topics of each machine/broker. To sum up the number of message for a topic,you will have to go on jconsole of each broker of your cluster and get the count of MessageIn from each Mbean "kafka.se

Re: Does Kafka stores Producer/Consumer Details

ikanth/netdb11/netdb11papers/netdb11-final12.pdf > > Thanks, > > Jun > > > On Thu, Oct 24, 2013 at 11:13 PM, Monika Garg > wrote: > > > Hi, > > > > > > > > I am wondering does metadata of Kafka stores producer/consumer > information. >

Please explain the behaviour of bin/kafka-list-topic.sh command

Hi My zookeeper node in Kafka is storing the data for a partition like this: { "controller_epoch":3, "isr":[ 1, 3 ], "leader":1, "leader_epoch":1, "version":1 } So what I am thinking is the command *bin/kafka-list-topic.sh *uses the above data from zookeeper in fetching the its output* + *some a

Does Kafka stores Producer/Consumer Details

Hi, I am wondering does metadata of Kafka stores producer/consumer information. As I start any producer,I provide a broker-list to it.But producer must be connecting to one of the broker to send data(as per Im gettiing the things).Similarly I can start more producer by giving a broker-list in t

Re: kafka monitoring

It is perBroker. It gives the count of messages that a broker has. As per my understanding to get it at cluster level,the messages count at each broker needs to be summed up. On Thu, Oct 24, 2013 at 2:19 PM, Kane Kane wrote: > I see this MBean: > "kafka.server":name="AllTopicsMessagesInPerSec",

Re: Any detail for Kafka Mbeans/yammer-Metrics obtained from JMX

e indicated leader brokers (1 and 2). > > > > I am getting some more Mbeans which are not mentioned here- > > http://kafka.apache.org/documentation.html#monitoring > > > Kafka exposes a *lot* of mbeans - not all of them are listed on that > page that you are referring

Re: Any detail for Kafka Mbeans/yammer-Metrics obtained from JMX

ot;ReplicaFetcherThread-0-2-host_moni-192-168-145-176-port_9092-BytesPerSec" The Mbeans (2),(3) and (4) are getting generated on my 3rd broker - host_moni-192.168.145.67,kafka running on same port-9092.Any reason for these. Any help is really appreciable. On Tue, Oct 22, 2013 at 11:01 PM, Monik

Re: Any detail for Kafka Mbeans/yammer-Metrics obtained from JMX

ntroller. > In general, having ISR shrink rate stay at 0 is a good thing since it means > that the followers can always keep up. > > Thanks, > > Jun > > > On Tue, Oct 22, 2013 at 12:55 AM, Monika Garg > wrote: > > > Thanks a lot Jun. > > > > Now

Re: Leader = -1 is coming when broker goes down

Thanks Neha. On Tue, Oct 22, 2013 at 7:39 PM, Neha Narkhede wrote: > You have picked replication factor of 1, so when you bring down the only > replica which can be the leader, it cannot be moved elsewhere. So we set it > to -1. > > Thanks, > Neha > On Oct 22, 2013 1

Re: Any detail for Kafka Mbeans/yammer-Metrics obtained from JMX

e producer/consumer requests.what is no of partitions=100 for 10 topics.There will be time delay,so other brokers should also be elected as leader after they are started. Thanks. On Tue, Oct 22, 2013 at 1:25 PM, Monika Garg wrote: > Thanks a lot Jun. > > Now im getting the values in ISR expa

Re: Any detail for Kafka Mbeans/yammer-Metrics obtained from JMX

s clearer? > > Thanks, > > Jun > > > On Mon, Oct 21, 2013 at 9:43 AM, Monika Garg wrote: > > > Thanks for replying Neha.It helped me a lot in getting the things more > > clear. > > I just have some doubts for the value of below Mbeans,please have a look:

Leader = -1 is coming when broker goes down

Hi I have made 2 nodes kafka cluster and created a topic with 3 partitions and 1 replica. the partitions got distributed among 2 brokers. then one of my broker(broker1) goes down and due to this the partition(partition2) for which broker1 was acting as a leader,now showing me the leader value as

Re: Any detail for Kafka Mbeans/yammer-Metrics obtained from JMX

erty(2) be always be zero? Thanks. On Mon, Oct 21, 2013 at 8:35 PM, Neha Narkhede wrote: > You may find this useful - > http://kafka.apache.org/documentation.html#monitoring > > Let us know how we can improve the documentation further. > > Thanks, > Neha > > > O

Any detail for Kafka Mbeans/yammer-Metrics obtained from JMX

Hi, I am getting so many Mbeans from My JMX like kafka.server,kafka.network etc. There are of different types and have a lot of attributes. I am trying to get when the attribute value for any Mbean is getting changed. But no helpful docs I am getting for it on google. Even the terms used like

Re: getting error in fetching topic metadata,when giving message to console producer in Kafka cluster

; > On Thu, Oct 17, 2013 at 1:51 AM, Monika Garg wrote: > > > Hi, > > > > I had 3 nodes Kafka cluster with default.replication.factor=3 in all the > > three nodes. > > I create a topic "topic1" with 7 partitions and with rep-factor = 3,my > lis

getting error in fetching topic metadata,when giving message to console producer in Kafka cluster

Hi, I had 3 nodes Kafka cluster with default.replication.factor=3 in all the three nodes. I create a topic "topic1" with 7 partitions and with rep-factor = 3,my list topic command gave below o/p: topic: topic1partition: 0leader: 1replicas: 3,2,1isr: 1,2,3 topic: topic1partitio

Re: Kafka and Zookeeper node removal from two nodes Kafka cluster

. > > As for ZK, you can always use the full connection string. ZK will pick live > servers to establish connections. > > Thanks, > > Jun > > > On Tue, Oct 15, 2013 at 3:46 AM, Monika Garg wrote: > > > I have 2 nodes kafka cluster with default.replication.fac

Kafka and Zookeeper node removal from two nodes Kafka cluster

I have 2 nodes kafka cluster with default.replication.factor=2,is set in server.properties file. I removed one node-in removing that node,I killed Kafka process,removed all the kafka-logs and bundle from that node. Then I stopped my remaining running node in the cluster and started again(default.

Re: Messages from producer are immediately going to /tmp/logs in kafka

sk before shutting down. If you want to force data > > loss I think you need to yank the plug on the machine immediately after a > > write but prior to both an application level fsync and the OS's own flush > > policy. > > > > -Jay > > > > -Jay > >

Re: Messages from producer are immediately going to /tmp/logs in kafka

shutdown? On Fri, Oct 11, 2013 at 8:56 PM, Jun Rao wrote: > Those messages could still be in file system pagecache and may not be > flushed to disks. > > Thanks, > > Jun > > > On Thu, Oct 10, 2013 at 11:20 PM, Monika Garg > wrote: > > > Hi, > > > >

Fwd: Messages from producer are immediately going to /tmp/logs in kafka

Hi, In kafka-0.8 there are three important properties given for log.flush.interval.messages=1 log.flush.interval.ms=90 log.flush.scheduler.interval.ms=90 I have set the above properties as I have mentioned above.Then I started Kafka Console Producer given with kafka bundle-0.8 and

Re: kafka.common.UnknownTopicOrPartitionException

Hi Snehalata state-change log are automatically created if you are using kafka-0.8 version. On Wed, Oct 9, 2013 at 10:42 AM, Snehalata Nagaje < snehalata.nag...@harbingergroup.com> wrote: > Actually state-change log setting is not on. So could not find the > state-change log. > > ---

Kafka-0.8.0-beta1-src Has ObjectName starting with Double Quotes

Hi, Kafka-0.8.0-beta1-src is having doublequotes in the start of Objects name obtained from jConsole.Due to this I am not able to use jmxTrans to monitor my kafka-0.8.0 cluster. Please help in solving the issue. Regards Monika Garg Associate Software Engineer Impetus Infotech (India) Pvt