Info regarding kafka topic

Hi, I have a 3 node Kafka Broker cluster. I have created a topic and the leader for the topic is broker 1(1001). And the broker got died. But when I see the information in zookeeper for the topic, I see the leader is still set to broker 1 (1001) and isr is set to 1001. Is this a bug in kafka, as no

users@kafka.apache.org

Hi All, I am facing issue with kafka kerberoized cluster. After following the steps how to enables SASL on kafka by using below link. http://docs.confluent.io/2.0.0/kafka/sasl.html After this,when i start the kafka-server I am getting below error. [2016-04-12 16:59:26,201] ERROR [KafkaApi-1001]

users@kafka.apache.org

t configures authentication and authorization > and includes a vagrant setup that you can test: > > > http://www.confluent.io/blog/apache-kafka-security-authorization-authentication-encryption > > Ismael > > On Wed, Apr 13, 2016 at 1:06 AM, BigData dev > wrote: > > > Hi A

Reg: Kafka With Kerberos/SSL [Enhancement to add option, Need suggestions on this]

Hi All, When Kafka is running on kerberoized cluster/ SSL. Can we add an option security.protocol. So, that user can given PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL. This will be helpful during running console producer and console consumer. ./bin/kafka-console-producer.sh --broker-list --topic --

Re: kafka-console-consumer using which API?

By default it uses old conusmer. When you specify --new-consumer it uses new consumer. Regards, Bharat On Mon, Apr 25, 2016 at 1:07 PM, Ramanan, Buvana (Nokia - US) < buvana.rama...@nokia.com> wrote: > Does kafka-console-consumer.sh utilize New Consumer API or Old Consumer > API? > >

Reg: Kafka-Acls

Hi, When I run the command /bin/kafka-acls.sh --topic permissiontopic --add --allow-host {host} --allow-principal User:dev --operation Write --authorizer-properties zookeeper.connect={host:port} I am getting output as acls are set. But when i check under zookeeper using below command, it is not

Re: [DISCUSS] Java 8 as a minimum requirement

+1. Thanks, Bharat On Thu, Jun 16, 2016 at 1:59 PM, Adam Kunicki wrote: > +1 > > Adam Kunicki > StreamSets | Field Engineer > mobile: 415.890.DATA (3282) | linkedin > < > https://mailtrack.io/trace/link/3e560367e0508b2f285512f39bd070275e70f571?url=http%3A%2F%2Fwww.adamkunicki.com&signature=aab

Reg: SSL setup

Hi, Can you please provide information on Self signed certificate setup in Kafka. As in Kafka documentation only CA signed setup is provided. http://kafka.apache.org/documentation.html#security_ssl As because, we need to provide parameters trustore, keystore during configuration. Or to work wit

Re: Kafka ACLs CLI Auth Error

Hi, I think jaas config file need to be changed. Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/etc/security/keytabs/kafka.service.keytab" storeKey=true useTicketCache=false serviceName="zookeeper" principal="kafka/hostname.abc@abc.c

Reg: DefaultParititioner in Kafka

Hi All, In DefaultPartitioner implementation, when key is null, we get the partition number by modulo of available partitions. Below is the code snippet. if (availablePartitions.size() > 0) { int part = Utils.toPositive(nextValue) % availablePartitions.size(); return availablePartitions.get(part).

Re: [ANNOUNCE] New committer: Jiangjie (Becket) Qin

Congratulations Becket!! Thanks, Bharat On Mon, Oct 31, 2016 at 11:26 AM, Jun Rao wrote: > Congratulations, Jiangjie. Thanks for all your contributions to Kafka. > > Jun > > On Mon, Oct 31, 2016 at 10:35 AM, Joel Koshy wrote: > > > The PMC for Apache Kafka has invited Jiangjie (Becket) Qin t

Reg: Kafka ACLS

Hi, I have a question, can we use Kafka ACL's with only SASL/PLAIN mechanism. Because after I enabled, still I am able to produce/consume from topics. And one more observation is in kafka-_jaas.conf, there is no client section, will get an WARN as below, as we dont have this kind of mechanisim wit

Reg: Kafka Kerberos

Hi, I am using Kafka 0.10.1.0 and kerberozied cluster. Kafka_jaas.conf file: Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/etc/security/keytabs/kafka.service.keytab" storeKey=true useTicketCache=false serviceName="zookeeper" principal=

Reg: Kafka HDFS Connector with (HDFS SSL enabled)

Hi, Does Kafka HDFS Connect work with HDFS (SSL). As I see only properties in security is hdfs.authentication.kerberos, connect.hdfs.keytab,hdfs.namenode.principal as these properties are all related to HDFS Kerberos. As from the configuration and code I see we pass only Kerberos parameters, not

Re: Reg: Kafka HDFS Connector with (HDFS SSL enabled)

S or WebHDFS to access > HDFS, which might motivate you to configure SSL. Are you trying to > configure one of these? > > best, > Colin > > > On Wed, Feb 15, 2017, at 11:03, BigData dev wrote: > > Hi, > > > > Does Kafka HDFS Connect work with HDFS

Re: any way to manually assign listeners to kafka partitions ?

There is a method is consumer is used for getting all partitions for a topic List https://kafka.apache.org/0100/javadoc/org/apache/kafka/common/PartitionInfo.html> > *partitionsFor

Need Permissions for creating KIP in Kafka

Hi, Could you please provide permission for creating KIP in Kafka. username: bharatv email: bigdatadev...@gmail.com

[DISCUSS] KIP-156 Add option "dry run" to Streams application reset tool

Hi All, I want to start a discussion on this simple KIP for Kafka Streams reset tool (kafka-streams-application-reset.sh). https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=69410150 Thank you, Matthias J Sax for providing me Jira and info to work on. Thanks, Bharat

Re: producer and consumer sample code

Hi Adaryl, There are samples of java producer/consumer in Apache Kafka source code repo. https://github.com/apache/kafka/tree/trunk/examples/src/main/java/kafka/examples And also there is an IBM Hadoop Dev Blog to get started with Kafka. The code sample is attached to the blog. https://develope

[VOTE] KIP-156 Add option "dry run" to Streams application reset tool

Hi, Everyone, Since this is a relatively simple change, I would like to start the voting process for KIP-156: Add option "dry run" to Streams application reset tool https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=69410150 The vote will run for a minimum of 72 hours. Thanks,

Re: [VOTE] KIP-156 Add option "dry run" to Streams application reset tool

Eno > > > On May 9, 2017, at 8:41 PM, BigData dev wrote: > > > > Hi, Everyone, > > > > Since this is a relatively simple change, I would like to start the > voting > > process for KIP-156: Add option "dry run" to Streams application reset > tool &

Re: [VOTE] KIP-156 Add option "dry run" to Streams application reset tool

> > > On May 9, 2017, at 8:41 PM, BigData dev wrote: > > > > Hi, Everyone, > > > > Since this is a relatively simple change, I would like to start the > voting > > process for KIP-156: Add option "dry run" to Streams application reset > tool &

Re: [VOTE] KIP-156 Add option "dry run" to Streams application reset tool

> > > > > > > > On Tue, May 9, 2017 at 7:57 PM Jay Kreps wrote: > > > > > > > +1 > > > > On Tue, May 9, 2017 at 3:41 PM BigData dev > > > > wrote: > > > > > > > > > Hi, Everyone, > > > > > >

Re: [VOTE] KIP-156 Add option "dry run" to Streams application reset tool

> > > > > > > > On Tue, May 9, 2017 at 7:57 PM Jay Kreps wrote: > > > > > > > +1 > > > > On Tue, May 9, 2017 at 3:41 PM BigData dev > > > > wrote: > > > > > > > > > Hi, Everyone, > > > > > >

Reg: [VOTE] KIP 157 - Add consumer config options to streams reset tool

Hi All, Given the simple and non-controversial nature of the KIP, I would like to start the voting process for KIP-157: Add consumer config options to streams reset tool *https://cwiki.apache.org/confluence/display/KAFKA/KIP+157+-+Add+consumer+config+options+to+streams+reset+tool

Re: `key.converter` per connector

Hi, key.converter and value.converter can be overridden at connector level. This has been supported from Kafka 0.10.1.0 For more info refer to this KIP https://cwiki.apache.org/confluence/display/KAFKA/KIP-75+-+Add+per-connector+Converters Thanks, Bharat On Tue, May 16, 2017 at 1:39 AM, Nicol