Re: Using ACLs without Kerberos

Hi, Kafka default authorizer is used with secure authenticated channels (SSL,SASL,SCRAM). For plain text (non-secure) channels, the principal will be always ANONYMOUS. Here you can authorize by ip-address. It's adviced to run on secure channels. you can try SASL/PLAIN or SCRAM mechanisms with/wit

Re: How to debug - NETWORK_EXCEPTION

I was never able to debug this exception. I, unfortunately, moved to Apache Kafka 10.2.1 from Confluent 3.2.1 and this issue went away. On Wed, Aug 30, 2017 at 11:46 AM, Saravanan Tirugnanum wrote: > Hi > > Were you able to figure out this issue? Any clue ? > > Regards > Saravanan > > > On Wedne

Re: Strange Kafka throughput issues

Thanks Boris, It's already using SSD. Yes, I understand that about EBS. But I don't think that explains the throughput I was getting with 1K message size (300K msgs/sec @ 300MB/sec) vs 48K message size (1160 msgs/sec @ 53MB/sec). and the fact that nothing seems to affect the throughput by ch

Using ACLs without Kerberos

Hi, We are evaluating how to put authorization in place for Kafka (around topics, mostly). Is it a good idea to do this without Kerberos? I was testing if a non-admin principal (OS user) can modify (add/remove) ACLs and it seems like it's possible. If this is right behavior, it's insecure and unus

Re: Strange Kafka throughput issues

Can you please do 2 additional tests: 1. Take SSD based instances and put Kafka storage on SSD. 2. Run storage benchmark with the same block size. What throughput do you receive? What EBS storage type do you use? Remember, EBS network is shared with the instance network so you'd get at most half o

Re: How to debug - NETWORK_EXCEPTION

Hi Were you able to figure out this issue? Any clue ? Regards Saravanan On Wednesday, August 9, 2017 at 11:51:19 PM UTC-5, Raghav wrote: > > Hi > > I am sending very small 32 byte message to Kafka broker in a tight loop > with 250ms sleep. I have one broker, 1 partition, and replication factor

Re: Strange Kafka throughput issues

Anyone has thought on this? Anyone producing messages of this size and can you share your config? Or I wonder if it would be inappropriate use case to send record size of 48K at a rate of 400 MB/sec. On Mon, Aug 28, 2017 at 2:59 PM, Sunny Kim wrote: > Hello, > > First here's what I have: > >

Kafka with SSL errors

I am trying to configure a brand new Kafka cluster/sandbox with SSL, but I keep getting errors. I apologize for the length of this email, but I've never worked with keystores/certificates before, so while I'm trying to follow the directions here ( http://kafka.apache.org/documentation.html#security

hi//Re: 答复: NoAuth for /controller

hi,man i have an problem when using kafka, when i set acl in zookeeper on /, then i try to restart kafka, but it doesn't work: org.I0Itec.zkclient.exception.ZkException: org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode = NoAuth how to settle this problem. thanks

Commnication Endpoint implementation for Connector.

Hey Guys, We have got a use case of triggering the connector via the Rest endpoint, I would rather prefer to build a connector with the generic pluggable communication endpoint. I would be interested to know if anyone is the community had such use case and have implemented similar connector. I did

Re: kafka consumer rebalance

thanks manikumar, this helped us a lot. On Wed, Aug 30, 2017 at 11:33 AM, Manikumar wrote: > In old consumer, group coordination is based on zookeeper and new consumer > uses inbuilt > (not depending on ZK) group coordinator. As of now, auto-migration > of migration of old consumers > to new co

Is CommitFailedException thrown from commitSync with offsets because an offset is out of range?

Hello Apache Kafka community, org.apache.kafka.clients.consumer.CommitFailedException javadoc and especially exception message suggests that the exception can be thrown only due to consumer re-balancing, KafkaConsumer instance not owning TopicPartition for the consumer group of the offset being co