UNKNOWN_TOPIC_OR_PARTITION for kafka producer

Hi All, I have SSL from client to broker,SSL again for broker to borker communication and kerberos from broker to ZK The topics exists when i do describe from ZK Cli. Authorization log clearly says, it can allow principal to describe and write to the topic. When producing the message using kafka

Controller connection to broker was unsuccessfull

Hi All, I am using SSL communication between the Kafka brokers. When i am sending the data, i am getting following error: WARN [Controller-3-to-broker-2-send-thread], Controller 3' connection to the broker to the broker :5010(id 2 rack:null) was unsuccessful (kafka.controller.RequestSendThread

Kafka Custom Authentication & Authorization

Hi All, To Authenticate & Authorize the producer and consumers, I want to integrate with third party Entitlement manager. Is there reference implementation ? Any link for doc or comments I appreciate. Regards Bhanu

Kafka Monitoring..

Hi All, What is the simplest way of monitoring the metrics in kaka brokers? Is there any opensource available? Any help in this regards is appreciated. Regards Bhanu

Re: Kafka JVM heap limit

0) To: users@kafka.apache.org Cc: > ja...@scholz.cz Subject: Re: Kafka JVM heap limit > In addition, in my experience, a memory heap > 8 GB leads to long GC pauses > which causes the ISR statuses to constantly change, leading to an unstable > cluster. > > --John > > On Wed, No

Re: Kafka JVM heap limit

Meaning, already read the doc, but couldn't relate, having large Heap for JVM will not help Now it all make sense. Many thanks. Bhanu On Wed, Nov 8, 2017 at 2:34 PM, chidigam . wrote: > Hi Jakub, > Thank you very much. I have read this concept I/O cache in design section. > But co

Re: Kafka JVM heap limit

tem instead of storing messages in the JVM memory. So > the requirements for the heap memory can be fairly small. The design > section of the documentation describes the details: > http://kafka.apache.org/documentation/#design > > Jakub > > On Wed, Nov 8, 2017 at 9:30 AM, chidig

Kafka JVM heap limit

Hi All, I have basic question on Kafka JVM configuration, in most of forums I have seen max heap as 8GB. Why it is not recommended beyond that. Is there any design limitation ? Any help in this regards is highly appreciated. Regards Bhanu

Kafka Start Up and Shutdown

Hi All, I am trying to understand different configurations to apply on my cluster. I find the num.recovery.threads.per.data.dir=1,bit interesting. In most of configuration example I find in the google, the value set as 1. Everyone wants quick start and recovery, to achieve the same can I increase t

Linux OS level settings Kafka broker

Hi All, I am trying to understand, what can max. no.of threads running on busy kafka broker. If I am missing any other configuration to consider at OS level please let me know. Thanks in advance. bhanu

Re: No. of Kafk Instances in Single machine

Best Jan > > > On 06.11.2017 12:23, chidigam . wrote: > >> Hi All, >> Let say, I have big machine, which having 120GB RAM, with lot of cores, >> and very high disk capacity. >> >> How many no. of kafka instances are recommended? Is there any general >>

No. of Mirroring instances

Hi All, How can I calculate the no. of Mirroring instance required to run, to keep both the cluster nearly in sync ? For example, Cluster1 have 3 node, and Cluster 2 have 3 nodes. Do I have to run 3 mirror instance ? Regards Bhanu

No. of Kafk Instances in Single machine

Hi All, Let say, I have big machine, which having 120GB RAM, with lot of cores, and very high disk capacity. How many no. of kafka instances are recommended? Is there any general principle I can apply, to calculate optimal no. Any help in this regards is highly appreciated. Regards Bhanu

Re: Kafka Security SSL and SASL_PLAINTEXT

One more point I have missed out,I am using just one zk and one kafka instance. Regards Bhanu On Fri, Nov 3, 2017 at 10:51 PM, chidigam . wrote: > Hi Jakub, > I believe there is some thing wrong with inter broker communication. In > producer my send call just blocks. > As told you

Re: Kafka Security SSL and SASL_PLAINTEXT

ogs from your producer? Is it just the producer what is > not working? Or is the producer not working because the Interbroker > communication doesn't work? > > J. > > On Fri, Nov 3, 2017 at 6:02 PM, chidigam . wrote: > > > Hi Jakub, > > Thanks for respondi

Re: Kafka Security SSL and SASL_PLAINTEXT

/:9092,REPLICATION://:19092 > listener.security.protocol.map=SSL:SSL,REPLICATION:SASL_PLAINTEXT > inter.broker.listener.name=REPLICATION > > (plus all the SSL and Kerberos configuration) > > Jakub > > On Fri, Nov 3, 2017 at 4:48 PM, chidigam . wrote: > > > Hi All, > &g

Kafka Security SSL and SASL_PLAINTEXT

Hi All, I am new to Kafka and trying to understand possible security combinations. I want to do Client authentication and authorization only with SSL. Inter broker communication should be with Kerberos with out SSL. Is this possible? Can anyone help me with configurations. Regards Bhanu