Consumer blocked when auth failed.

Hi, I was integrating Kafka with Spark, using DirectStream, when my authentication fail, the stream just blocked. No log, no exceptions were thrown. Could some one help to address such situtation

Re: Consumer blocked when auth failed.

exception in the main thread instead of just blocking. 2017-07-20 14:11 GMT+08:00 Jason Gustafson : > What log level do you have configured? You might bump up to DEBUG or TRACE > and see if anything stands out. > > -Jason > > On Tue, Jul 18, 2017 at 7:59 PM, 陈江枫 wrote: >

Kafka latency optimization

Hi, everyone My kafka version is 0.10.2.1. My service have really low qps (1msg/sec). And our requirement for rtt is really strict. ( 99.9% < 30ms) Currently I've encounter a problem, when kafka run for a long time, 15 days or so, performance start to go down. 2017-10-21 was like Time .

Re: Kafka Custom Authentication & Authorization

You can implement your own authenticator, check SaslAuthenticator , and your own authorizer, check SimpleAuthorizer 2017-11-12 9:39 GMT+08:00 chidigam . : > Hi All, > To Authenticate & Authorize the producer and consumers, I want to > integrate with third party Entitlement manager. Is there ref

Long log appending time

Hi, all I'm trying to optimize the long tail latency in kafka. My OS is centos 4.3 kernel 2.6.32. Java version 1.8.0_45 kafka version 0.10.2. The disk is sata HDD. My log: [2017-12-22 15:29:58,156] TRACE [ReplicaFetcherThread-2-1], Follower 2 has replica log end offset 10428 for partition kafka-

Re: Long log appending time

by the way, I have 3 brokers, the producing rate is about 250 messages/sec.. 2017-12-22 16:20 GMT+08:00 陈江枫 : > Hi, all > > I'm trying to optimize the long tail latency in kafka. > My OS is centos 4.3 kernel 2.6.32. Java version 1.8.0_45 > kafka version 0.10.2. > The d

Schema registry

Hi, I'm new to Kafka, and I would like to use schema registry to manager the schema of my topic. The schema I've created: curl -X POST -i -H "Content-Type: application/vnd.schemaregistry.v1+json" --data '{ "schema": "{\"type\": \"record\",\"name\": \"Customer\", \"fields\": [ { \"type\": \"int\",

Re: How to choose one broker as Group Coordinator

when a consumer join a group, selection will be triggered, and then rebalance. 2017-02-15 17:59 GMT+08:00 Yuanjia : > Hi all, > Group Coordinator can be different for different consumer groups,When > a consumer wants to join a group,how to choose the Group Coordinator? > > Thanks, > Yuanjia L

kafka authenticate

Hi, all I'm trying to modify kafka authentication using our own authenticating procedure, authorization will stick to kafka's acls . Does every entry which fetches data from certain topic need to go through authentication? ( Including KafkaStreams, replica to leader ,etc.)