ssl.endpoint.identification.algorithm and performance

2020-07-08 Thread Gérald Quintana
Hello, We noticed that setting ssl.endpoint.identification.algorithm to empty (on both client and broker side) we got a big performance improvement in terms of throughput. As far as I understand, this is related to the SSL connection doing a DNS lookup to check that the host matches the certificat

Re: Broker thread pool sizing

2020-06-19 Thread Gérald Quintana
should double that if TLS is enabled in the broker. > > Thanks, > > -- Ricardo > On 6/19/20 8:56 AM, Gérald Quintana wrote: > > Hello, > > How do you size Kafka broker thread pools, in particular num.io.threads (8 > by default) and num.network.threads (3 by

Broker thread pool sizing

2020-06-19 Thread Gérald Quintana
Hello, How do you size Kafka broker thread pools, in particular num.io.threads (8 by default) and num.network.threads (3 by default) depending on the number of CPU cores available on the host? Regards, Gérald

InvalidRecordException: Inner record LegacyRecordBatch

2020-05-14 Thread Gérald Quintana
Hello, Since we upgraded from Kafka 2.3 to 2.4 we have big amount logs telling: [2020-05-14 19:34:27,623] ERROR [ReplicaManager broker=1] Error processing append operation on partition ourtopic-1 (kafka.server.ReplicaManager) org.apache.kafka.common.InvalidRecordException: Inner record LegacyReco

Metrics in Kafka Connect

2020-04-30 Thread Gérald Quintana
Hello, We developed a custom Kafka Connect implementation for a specific need, and we would like to monitor its internals (request latency and rate, pool usage). Is it possible to publish custom metrics using the Kafka client metric framework (org.apache.kafka.common.metrics.*) . We would lik

Mirror Maker 2 internal topics

2020-02-20 Thread Gérald Quintana
Hello, I am prototyping Kafka replication with Mirror Maker 2. At the beginning, I had hard times with org.apache.kafka.connect.errors.ConnectException: Error while attempting to create/find topic(s) 'mm2-offsets.dst.internal'... Caused by: java.util.concurrent.ExecutionException: org.apache.kafka

Kafka SSL and multiple domain names

2019-02-14 Thread Gérald Quintana
Hello, We need to have the same Kafka cluster bound to multiple DNS aliases/domain names. However, for some poor reason, we can't have a single SSL certificate with subject alt names matching all DNS aliases. Is it possible to use different SSL certs depending on the hostname used by the client?

Logging authentication/authorization failures

2018-05-31 Thread Gérald Quintana
Hello, I am using SASL Plaintext authentication and ACLs. I'd like to be able to detect potential security attacks on Kafka broker Is it possible to log, on broker side, authentication failures (wrong password) and authorization failures (not granted)? I read this blog post https://www.confluent