Question about Kafka connect error handling and improved logging

Hi Team, I was trying to leverage some enhancements in Kafka connect in 2.0.0 release as specified by this KIP https://cwiki.apache.org/confluence/display/KAFKA/KIP-298%3A+Error+Handling+in+Connectand I came across this good blog post by Robin https://www.confluent.io/blog/kafka-connect-deep-dive-

Re: [VOTE] 2.2.1 RC1

Since there is no vote on this RC yet, I'll extend the deadline to Monday, May 20, at 9:00 am. Thanks in advance for checking / testing / voting. --Vahid On Mon, May 13, 2019, 20:15 Vahid Hashemian wrote: > Hello Kafka users, developers and client-developers, > > This is the second candidate

Re: Question about Kafka TLS

Hi Thomas, We recently fixed a bug https://issues.apache.org/jira/browse/KAFKA-8191 , which allows users to configure their own KeyManager, TrustManager. One can implement these KeyManagers and pass them as configs and these Keymanagers can make a call to service to fetch a certif

Re: Help - Updating Keystore Dynamically - KAFKA-6810

Yes, it is still relevant — unless you’ve enabled SSL for inter-broker communication and you are trying to update the truststore associated with that listener. You should use the kafka-configs command to set the dynamic config value: https://kafka.apache.org/21/documentation.html#dynamicbrokerc

Question about Kafka TLS

Hi, I have a question about how TLS config at Kafka client side. Based on the official document, if clients want to enable TLS, they must put ssl.truststore.location in the client config in where there is a JKS file to hold the trust store. My question is that is this config mandatory? Is there

Re: Help - Updating Keystore Dynamically - KAFKA-6810

I sent another email that I am looking to dynamically update SSL truststore, and not keystore. Would that be still relevant? Thanks. On Thu, May 16, 2019 at 2:54 PM Peter Bukowinski wrote: > It’s my understanding that dynamic configuration requires you to write > znodes, e.g. /config/brokers/ssl

Re: Help - Updating Keystore Dynamically - KAFKA-6810

It’s my understanding that dynamic configuration requires you to write znodes, e.g. /config/brokers/ssl.keystore.location. I believe you can use the same path. Brokers should be watching that path and if a node is added or updated the config values will be read in and loaded over existing values

Re: Help - Updating SSL Truststore Dynamically - KAFKA-6810

I edited the email subject since it was not correct. Thanks. On Thu, May 16, 2019 at 2:08 PM Darshan wrote: > Hi > > I am testing out Kafka 2.2.0 and was hoping to test out "Enable dynamic > reconfiguration of SSL truststores" > https://issues.apache.org/jira/browse/KAFKA-6810. But unfortunately

Help - Updating Keystore Dynamically - KAFKA-6810

Hi I am testing out Kafka 2.2.0 and was hoping to test out "Enable dynamic reconfiguration of SSL truststores" https://issues.apache.org/jira/browse/KAFKA-6810. But unfortunately I could not get it work. Please find the server.properties. Just wondering if we need an change of config. Please advis

Re: Multiple state store files

You might want to tune RocksDB as an alternative: https://github.com/facebook/rocksdb/wiki/RocksDB-Tuning-Guide You can pass in configs via `StreamsConfig` using `RocksDBConfigSetter` object. -Matthias On 5/16/19 6:24 PM, Parthasarathy, Mohan wrote: > Thanks. Unfortunately, this leads to lot of

Re: Multiple state store files

Thanks. Unfortunately, this leads to lot of open file descriptors in the application. Anyone else faced that situation ? I will increase the limit for now. Thanks Mohan On 5/16/19, 5:18 AM, "Matthias J. Sax" wrote: This is how RocksDB works, and it's normal. -Matthias O

Re: Multiple state store files

This is how RocksDB works, and it's normal. -Matthias On 5/15/19 10:55 PM, Parthasarathy, Mohan wrote: > Hi, > > I am seeing multiple state store files (.sst) created under each > directory. Is this normal ? If an application crashes > and comes back, would it cause this ? > > Thanks > Mohan

Re: SASL + SSL : authentication error in broker-to-broker communication

Kerberos and PLAIN_TEXT were acceptable here in server.properties as of 2017 https://docs.confluent.io/3.0.0/kafka/sasl.html since you're not getting SASL_256 or SASL_512 then a number of situations have to be checked did you install SCRAM LoginModule ? what are your SCRAM callback handlers (hi

Re: SASL + SSL : authentication error in broker-to-broker communication

the information in the cert comes from the original pfx if X509 Cert Subject/UID is incorrect you wont have a correspondence to username in key and you might need a new pfx "An X.509 certificate is a digital certificate that

Re: SASL + SSL : authentication error in broker-to-broker communication

Hello fellas, I tried to simplify my configuration by removing the ssl configuration to ease up debugging. It didn't change anything regarding SCRAM but with SASL_PLAIN, it worked out of the box. I found that solution good enough as I'm still using SSL. If it helps someone, here's my configurat

Broker does not stop gracefully with SIGTERM

Hello, hope you all are doing well, Am trying to stop gracefully a Broker with SIGTERM (-15). After almost 12 hours the process is still alive. I do not see any data/replication going in or out from this Broker. The following are the logs immediately after sending the SIGTERM signal to the process