Independent zookeep for kafka multi node

2017-08-23 Thread IT Consultant
Hi All Can I have three independent zookeepers tagged to three kafka brokers without any clustering or quorum ? Would it be a good idea ?

Multi-node deployment

2017-08-21 Thread IT Consultant
HI All , We are seeing following behavior , let me know if its expected or some configuration error . I have Apache Kafka running on three server on TLS protocol . They are clustered on ZK level . *Behaviour *, 1.* Unable to run only one instance* - When 2 out of 3 servers or instances goes dow

KStream Usage spikes memory consumption and breaks Kafka

2017-06-20 Thread IT Consultant
Hi All , Kafka instance is breaking down when used Kstream . It runs out of memory frequently resulting into service unavailabilty , Is it a good practice to use Kstream ? What other option must be tried to avoid such breakage ? If it's best pratice , how do we fine tune kafka to withhold load ?

Parsing Kafka logs using Logstash

2017-06-08 Thread IT Consultant
Hi All, Has anybody tried to parse Kafka logs using Logstash ? If yes , can you please share patterns used to parse . Thanks in advance.

Re: Kafka Over TLS Error - Failed to send SSL Close message - Broken Pipe

2017-06-07 Thread IT Consultant
Hi All , Thanks a lot for your help . A bug has been logged for said issue and can be found at , https://issues.apache.org/jira/browse/KAFKA-5401 Thanks again . On Sun, Jun 4, 2017 at 6:38 PM, Martin Gainty wrote: > > > From: IT Consultant <

Kafka Over TLS Error - Failed to send SSL Close message - Broken Pipe

2017-06-02 Thread IT Consultant
Hi All, I have been seeing below error since three days , Can you please help me understand more about this , WARN Failed to send SSL Close message (org.apache.kafka.common.network.SslTransportLayer) java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcherImpl.write0(Native Method

Failed to update metadata after 5000 ms

2017-05-10 Thread IT Consultant
Hi All, Currently , i am running TLS enabled multi-node kafka . *Version :* 2.11-0.10.1.1 *Scenario :* Whenever producer tries to produce around 10 records at once to kafka . *It gets failed to update metadata after 5000 ms error .* *Server.properties :* *[image: Inline image 1]* *Can you

Apache Kafka SSL Deployment

2017-04-12 Thread IT Consultant
Hi All , How can I avoid using password for keystore creation ? We are currently passing keystore password while accessing TLS enabled Kafka instance . I would like to use either passwordless keystore or avoid password for clients accessing Kafka .

Re: Kafka security

2017-04-11 Thread IT Consultant
fore creating a consumer or producer) > > System.setProperty("zookeeper.ssl.keyStore.password", password); > > martin > > > From: IT Consultant <0binarybudd...@gmail.com> > Sent: April 11, 2017 2:01 PM > To: users@kafka.apa

Kafka security

2017-04-11 Thread IT Consultant
Hi All How can I avoid using password for keystore creation ? Our corporate policies doesn't​allow us to hardcore password. We are currently passing keystore password while accessing TLS enabled Kafka instance . I would like to use either passwordless keystore or avoid password for cleint access

Re: Performance and encryption

2017-03-06 Thread IT Consultant
Hi Todd Can you please help me with notes or document on how did you achieve encryption ? I have followed data available on official sites but failed as I m no good with TLS . On Mar 6, 2017 19:55, "Todd Palino" wrote: > It’s not that Kafka has to decode it, it’s that it has to send it across

Re: Securing Multi-Node single broker kafka instance

2017-03-01 Thread IT Consultant
ver.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888 Generation of a key and certificate is enough or should i do anything on zookeeper front to make it work with kafka brokers ? Am i missing anything here? On Thu, Mar 2, 2017 at 3:08 AM, IT Consultant <0binarybudd...@g

Re: Securing Multi-Node single broker kafka instance

2017-03-01 Thread IT Consultant
c to enable SSL. > > -Harsha > > On Mar 1, 2017, 1:08 PM -0800, IT Consultant <0binarybudd...@gmail.com>, > wrote: > > Hi Harsha , > > > > Thanks a lot . > > > > Let me explain where am i stuck , > > > > i have three machines on whi

Re: Securing Multi-Node single broker kafka instance

2017-03-01 Thread IT Consultant
You need to make sure zookeeper hosts > and port are reachable from your broker nodes. > -Harsha > > On Wed, Mar 1, 2017 at 12:45 PM IT Consultant <0binarybudd...@gmail.com> > wrote: > > > Hi Team , > > > > Can you please help me understand , > >

Securing Multi-Node single broker kafka instance

2017-03-01 Thread IT Consultant
Hi Team , Can you please help me understand , 1. How can i secure multi-node (3 machine) single broker (1 broker ) Apache Kafka deployment secure using SSL ? i tried to follow instructions here but found pretty confusing . https://www.confluent.io/blog/apache-kafka-security-authoriz ation-authe