Re: Kafka write throughput tuning

One partition is one file on disk. This is why your throughput per partition will not go over 150MB/s. I can not explain why you do not get 300MB/s for two partition files on different disks. It could be related to your hardware. How fast are your writes using 2 dd processes? To get more thr

WindowStore fetch() ordering

Hello, I was reading: https://kafka.apache.org/0102/javadoc/org/apache/kafka/streams/state/ReadOnlyWindowStore.html#fetch(K,%20long,%20long) From both my experiments and the way

Re: WindowStore fetch() ordering

Hi Eric, For any given key the data will always be returned in increasing temporal order. So, yes, the most recent value for a key will be the last value. Thanks, Damian On Wed, 31 May 2017 at 09:51 Eric Lalonde wrote: > Hello, I was reading: > > > https://kafka.apache.org/0102/javadoc/org/apa

Kafka not starting up after Kerberos, Zookeeper conn error

Hello, I am currently stuck in enabling Kerberos and starting-up KafkaBroker afterwards (confluent oss 3.2.1). Zookeepers are running, but starting kafka (/opt/confluent/bin/kafka-server-start /etc/kafka/server.properties) gives me error: *[2017-05-31 12:21:10,523] ERROR SASL authentication fail

Re: KIP-162: Enable topic deletion by default

+1 > On May 27, 2017, at 9:27 PM, Vahid S Hashemian > wrote: > > Sure, that sounds good. > > I suggested that to keep command line behavior consistent. > Plus, removal of ACL access is something that can be easily undone, but > topic deletion is not reversible. > So, perhaps a new follow-up JI

Re: Kafka write throughput tuning

I'm no kafka expert and I've forgotten what little I learnt, however there must be a bottleneck somewhere. In your first instance of 3 partitions on 3 disks: - Are all partitions growing? - Are they growing about equally? - What else might be limiting aspect... -- what's the cpu activity like, pe

advertised.listeners

Hi Our Kafka broker has two IPs on two different interfaces. eth0 has 172.x.x.x for external leg eth1 has 1.x.x.x for internal leg Kafka Producer is on 172.x.x.x subnet, and Kafka Consumer is on 1.x.x.x subnet. If we use advertised.listeners=SSL://172.x.x.x:9093, then Producer can producer the

Re: advertised.listeners

Hello Darshan Have you tried SSL://0.0.0.0:9093 ? Rajani had suggested something similar to me a week back while I was trying to get a ACL based setup. Thanks. On Wed, May 31, 2017 at 8:58 AM, Darshan wrote: > Hi > > Our Kafka broker has two IPs on two different interfaces. > > eth0 has 172.x

Re: advertised.listeners

If you want to use different interfaces with the same security protocol, you can specify listener names. You can then also configure different security properties for internal/external if you need. listeners=INTERNAL://1.x.x.x:9092,EXTERNAL://172.x.x.x:9093 advertised.listeners=INTERNAL://1.x.x.x

Re: advertised.listeners

Thanks, Rajini and Raghav. Let me try this. This is helpful. On Wed, May 31, 2017 at 11:31 AM, Rajini Sivaram wrote: > If you want to use different interfaces with the same security protocol, > you can specify listener names. You can then also configure different > security properties for intern