SASL_PLAINTEXT impact on throughput and packets

Hi, We recently enabled timestamp and security features in our production clusters. We have 5 clusters which are smaller and 2 larger aggreagtion clusters which mirror data from the 5 clusters. The version of Kafka is 0.10.1.1. For security we enabled the brokers to have both PLAINTEXT and SASL_

Re: kafka-consumer-groups tool with SASL_PLAINTEXT

ot, please advise. > > Thanks. > --Vahid > > > > > From: Meghana Narasimhan > To: users@kafka.apache.org > Date: 07/24/2017 01:56 PM > Subject:kafka-consumer-groups tool with SASL_PLAINTEXT > > > > Hi, > What is the correct way to use th

kafka-consumer-groups tool with SASL_PLAINTEXT

Hi, What is the correct way to use the kafka-consumer-groups tool with SASL_PLAINTEXT security enabled ? The tool seems to work fine with PLAINTEXT port but not with SASL_PLAINTEXT. Can it be configured to work with SASL_PLAINTEXT ? If so what permissions have to enabled for it ? Thanks, Meghana

enabling 2 security protocols and ACL

Hi, I am working on enabling both PLAINTEXT and SASL_PLAINTEXT together on 2 ports so that clients have time to transition to SASL_PLAINTEXT without impact. The listeners work well, but I had some queries regarding ACL. If I enable the SimpleAuthorizer and add Read and Write ACL rules for the clie

Re: Zookeeper on same server as Kafka

been a bottleneck for >> us, so can't really say. >> >> Thanks, >> >> Michał >> >> On 02/06/17 21:47, Mohammed Manna wrote: >> >> Usually, the overhead comes when you have kafka and zookeeper doing the >> housekeeping (i.e. Disk IO) on

Zookeeper on same server as Kafka

Hi, What are the pros and cons of setting up Zookeeper on the same server as the Kafka broker ? Earlier offsets were being written to zookeeper which was a major overhead but with offsets being written to Kafka now, what other requirements should be taken into consideration for setting up Zookeeper

Re: Viewing timestamps with console consumer (0.10.1.1)

hana, > > > Please Try : kafka-console-consumer.sh --property print.timestamp=true > > > - Amrit > > > On Mon, Jan 30, 2017 at 10:20 PM, Meghana Narasimhan < > mnarasim...@bandwidth.com> wrote: > > > Hi, > > Is there a way to view message timestamp using console consumer ? > > > > Thanks, > > Meghana > > >

Viewing timestamps with console consumer (0.10.1.1)

Hi, Is there a way to view message timestamp using console consumer ? Thanks, Meghana

Re: Confluent platform for Kafka 0.10.1.1

confluent-delivers-upgrades-clients-kafka- > streams-brokers-apache-kafka-0-10-1-1/> > > -hans > > > > On Jan 25, 2017, at 7:40 AM, Meghana Narasimhan < > mnarasim...@bandwidth.com> wrote: > > > > Hi, > > As far as I understand, the current CP v3.1

Confluent platform for Kafka 0.10.1.1

Hi, As far as I understand, the current CP v3.1.1 supports Kafka 0.10.1.0. When is the next Confluent platform release supporting Kafka 0.10.1.1 planned for ? Thanks, Meghana

Re: timeindex file timestamp mismatch (0.10.1.0)

gSegment doesn't know the actual valid entries in the index and will > report those 0 values as errors. If you wait for the log segment to roll, > you should see the timeindex resized to 0 length. > > Thanks, > > Jun > > On Fri, Jan 20, 2017 at 8:20 AM, Meghana Narasim

Re: Leader imbalance issue

s a > leader imbalance. > > https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools# > Replicationtools-1.PreferredReplicaLeaderElectionTool > > Thanks, > Apurva > > > On Fri, Jan 20, 2017 at 8:29 AM, Meghana Narasimhan < > mnarasim...@bandwidth.com>

Re: timeindex file timestamp mismatch (0.10.1.0)

. > https://issues.apache.org/jira/browse/KAFKA-3163 > > Thanks > Sudev > > On Fri, Jan 20, 2017 at 9:50 PM Meghana Narasimhan < > mnarasim...@bandwidth.com> wrote: > > > Hi, > > I'm testing upgrading our cluster from 0.9.0.1 to 0.10.1.0 on 2 clusters > A > &

Re: Leader imbalance issue

Hi, Any input on this will be of great help . A rolling restart could fix the issue but not sure if thats the right way to do it. Thanks, Meghana On Wed, Jan 18, 2017 at 9:46 AM, Meghana Narasimhan < mnarasim...@bandwidth.com> wrote: > Hi, > > We have a 3 node cluster with 0.9.0

timeindex file timestamp mismatch (0.10.1.0)

Hi, I'm testing upgrading our cluster from 0.9.0.1 to 0.10.1.0 on 2 clusters A and B. I have upgraded only the inter.broker.protocol.version to 0.10.1.0. The log.message.format.version is still 0.9.0.1. I'm writing test data from a java producer to the upgraded cluster A. As expected the .timeinde

Leader imbalance issue

Hi, We have a 3 node cluster with 0.9.0.1 version. The controller is reporting that one of the brokers has a leader imbalance. One of the topics with 120 partitions has Broker 0 and 2 acting as leaders for all its partitions.None of the partitions have Broker 1 as their leader. So the controller

Kafka 0.9.0.1 ISR Shrink and Expand issue

Hi, Our cluster set up is as follows, with Version 0.9.0.1 (confluent package) we have 5 source clusters and 1 destination cluster. 5 instances of mirrormaker run on the destination cluster (one instance per source cluster, and each mirrormaker runs with num.streams=8) We have been seeing an issue

Re: Restoring Kafka data to one broker

Hi All, Any suggestions or inputs on this ? Any help would be greatly appreciated. Thanks, Meghana On Wed, Jun 1, 2016 at 3:01 PM, Meghana Narasimhan < mnarasim...@bandwidth.com> wrote: > Hi, > I have a 3 node cluster with kafka version 0.9.0.1 with many topics having > replicati

Restoring Kafka data to one broker

Hi, I have a 3 node cluster with kafka version 0.9.0.1 with many topics having replication factor 3 and isr 2. Node 0 is running multiple mirrormakers. Node 1 in this cluster has an issue and lost all data from disk (All kafka data logs got deleted from disk). Kafka broker is down but zookeeper is

Re: Brokers changing mtime on data files during startup?

I have seen this issue as well with 0.9. I also thought that it was because of the upgrade, but that doesn't seem to be it. But there were also a couple of instances when it didn't change the timestamps, so I was unable to pinpoint the exact root cause or steps and hence had not yet posted it here.

Re: Mirrormaker schema exception

der than the older broker involved in the > replication. > > Gwen > > On Mon, May 16, 2016 at 2:12 PM, Meghana Narasimhan > wrote: > > Hi, > > I came across the following mirrormaker issue today which caused it to > > shutdown. Don't see any indications of a

Mirrormaker schema exception

Hi, I came across the following mirrormaker issue today which caused it to shutdown. Don't see any indications of any issues in any of the other logs. Any insight on this error will be much appreciated. Kafka version is 0.9.0.1 (confluent platform 2.0.1). FATAL [mirrormaker-thread-0] Mirror maker

Re: Memory records is not writable in MirrorMaker

I have created a JIRA, https://issues.apache.org/jira/browse/KAFKA-3147 Thanks, Meghana On Tue, Jan 19, 2016 at 5:07 AM, Ismael Juma wrote: > Can you please file an issue in JIRA for this? > > Ismael > > On Tue, Jan 12, 2016 at 2:40 PM, Meghana Narasimhan < > mnarasim...

Re: Memory records is not writable in MirrorMaker

Hi, Came across a similar issue. We are running a 3 node cluster (kafka version 0.9) and Node 0 also has a few mirror makers running. When we do a rolling restart of the cluster, the mirror maker shuts down with the following errors. [2016-01-11 20:16:00,348] WARN Got error produce response with c

Re: Mirrormaker issue with Kafka 0.9 (confluent platform 2.0)

ssary (that JIRA fixes a data loss issue) or an accidental > > incompatibility. > > > > -Ewen > > > > On Thu, Dec 10, 2015 at 2:32 PM, Meghana Narasimhan < > > mnarasim...@bandwidth.com> wrote: > > > > > Hi, > > > > > > We

Mirrormaker issue with Kafka 0.9 (confluent platform 2.0)

Hi, We upgraded to the Confluent 2.0 platform today (we were earlier using the Kafka 0.8 without the confluent platform). With the latest Kafka 0.9 that is packaged with the platform we are facing issues starting the Mirror maker with multiple consumer config files. The original command that we w

Re: Kafka consumer offset checker hangs indefinitely

data dir, you can try deleting just __consumer_offsets data. > > Thanks, > Raja. > > > On Thu, May 14, 2015 at 10:46 AM, Meghana Narasimhan < > mnarasim...@bandwidth.com> wrote: > > > Hi Mayuresh, > > A few more inputs that I can provide at the moment after some

Re: Kafka consumer offset checker hangs indefinitely

yuresh > > On Mon, May 11, 2015 at 1:53 PM, Meghana Narasimhan < > mnarasim...@bandwidth.com> wrote: > > > Hi Mayuresh, > > A small update. The Kafka version I'm currently using is 2.10-0.8.2.1 > (not > > 2.11 as previously mentioned). The cluster looks f

Re: Kafka consumer offset checker hangs indefinitely

er: 0 Replicas: 0,1,2 Isr: 1,2,0 On Fri, May 8, 2015 at 12:52 PM, Meghana Narasimhan < mnarasim...@bandwidth.com> wrote: > Hi Mayuresh, > > Yes, the broker is up and accepting connections. Multiple consumers are > consuming off topics on the broker. > Also I am seein

Kafka consumer offset checker hangs indefinitely

Hi, I'm using the Kafka 8.2.1 version(kafka_2.11-0.8.2.1) and the consumer offset checker hangs indefinitely and does not return any results. I enabled the debug for tools and below is the debug statements as seen on the stdout. Any thoughts or inputs on this will be much appreciated. command used

Re: Kafka consumer offset checker hangs indefinitely

gt; of the brokers and get the offsetmanager for its group and then perform the > offset management. > > Thanks, > > Mayuresh > > On Fri, May 8, 2015 at 9:22 AM, Meghana Narasimhan < > mnarasim...@bandwidth.com> wrote: > > > Hi, > > I'm using the

Fwd: Kafka consumer offset checker hangs indefinitely

Hi, I'm using the Kafka 8.2.1 version(kafka_2.11-0.8.2.1) and the consumer offset checker hangs indefinitely and does not return any results. I enabled the debug for tools and below is the debug statements as seen on the stdout. Any thoughts or inputs on this will be much appreciated. command used