Re: data structures used by GlobalKTable, KTable

2020-05-22 Thread Matthias J. Sax
By default, RocksDB is used. Ie, the following are the same: builder.globalTable("topic"); builder.globalTable( "topic", Materialized.as(Stores.persistentKeyValueStore("store-name")) ); -Matthias On 5/22/20 4:31 AM, Pushkar Deole wrote: > Matthias, > > I can see that when the Materialized

Re: User ID / Password based authentication for Kafka Connect Source connector REST Interface

2020-05-22 Thread ashish sood
Thanks a lot Robin. You saved my weekend :) I noticed that we would need to store the password in text format. Similarly for SCRAM authentication I have also updated the kafka infrastructure credentials in text format. [producer.sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginMod

KafkaProducer Metadata versioning

2020-05-22 Thread Murilo Tavares
Hi I wonder how the KafkaProducer Metadata version works. I have some KafkaProducers in Production that started failing and will not recover after a topic was recreated (deleted and created again). Those are shared producers, so I stopped publication of that topic before the procedure, but I believ

Re: User ID / Password based authentication for Kafka Connect Source connector REST Interface

2020-05-22 Thread Robin Moffatt
See here: https://docs.confluent.io/current/security/basic-auth.html#basic-auth-kconnect -- Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff On Fri, 22 May 2020 at 15:42, wrote: > Hi Team, > > I have configured a Kafka Connect source connector on Distributed worker >

User ID / Password based authentication for Kafka Connect Source connector REST Interface

2020-05-22 Thread Ashish.Sood
Hi Team, I have configured a Kafka Connect source connector on Distributed worker and I am using the REST Interface for administering it like for starring/stopping/querying etc [curl https://host:port/connectors , curl https://host:port/connectors/https://host:port/connectors/%3cname>> etc]. I

Re: swagger/openapi spec for Kafka-Connect REST API?

2020-05-22 Thread Jun Wang
I am not aware of any swagger api spec for Kafka connect reset api. But following 2 resources have enough information for me. https://kafka.apache.org/documentation/#connect_rest https://docs.confluent.io/current/connect/references/restapi.html From: Miguel Silve

Re: swagger/openapi spec for Kafka-Connect REST API?

2020-05-22 Thread Miguel Silvestre
Hi, I'm also searching for this. If you find it, please let me know. Thank you. -- Miguel Silvestre On Mon, May 18, 2020 at 8:59 AM Aki Yoshida wrote: > I am looking for the openapi spec for Kafka-Connect API but I couldn't > find one. If it is available somewhere, could someone tell me where

Re: data structures used by GlobalKTable, KTable

2020-05-22 Thread Pushkar Deole
Matthias, I can see that when the Materialized parameter is provided with store-name, the store can be accessed through kafkaStreams.store(store-name). This returns ReadOnlyKeyValueStore which seems to hold the Map structure internally. So, the GlobalKTable API is just an abstraction and if I have

Re: Is that a bug?

2020-05-22 Thread sunil chaudhari
Simple.. Topic auto creation ON. As soon as it encounters producer and consumers request for some topic then it creates that topic automatically. This is very common question and problem people come across, if they are new. Same happened with me.😄 On Fri, 22 May 2020 at 3:28 PM, Jiamei Xie wrot

RE: Is that a bug?

2020-05-22 Thread Jiamei Xie
Hi Manna, Luke I used the default configure files without any modification. The default value for allow.auto.create.topics is true according to https://kafka.apache.org/documentation/#brokerconfigs. Thanks for your reply. -Original Message- From: M. Manna Sent: 2020年5月22日 16:50 To: us

Re: Is that a bug?

2020-05-22 Thread Luke Chen
Yes, for more specific, it's the property: *allow.auto.create.topics, *which default is set to true. On Fri, May 22, 2020 at 5:07 PM M. Manna wrote: > Hey Xie, > > On Fri, 22 May 2020 at 08:31, Jiamei Xie wrote: > > > Hi > > > > Kill all zookeeper and kafka process. Clear zookeeper and kafka da

Re: Is that a bug?

2020-05-22 Thread M. Manna
Hey Xie, On Fri, 22 May 2020 at 08:31, Jiamei Xie wrote: > Hi > > Kill all zookeeper and kafka process. Clear zookeeper and kafka data dir. > Restart zookeeper and kafka. If there are any active client. Topic used by > client will be auto-created. > > How to reproduce? > > > 1. Start zookeep

User ID / Password based authentication for Kafka Connect Source connector REST Interface

2020-05-22 Thread Sood Ashish (UK) - Cognizant
Hi Team, I have configured a Kafka Connect source connector on Distributed worker and I am using the REST Interface for administering it like for starring/stopping/querying etc [curl https://host:port/connectors , curl https://host:port/connectors/https://host:port/connectors/%3cname>> etc].

Is that a bug?

2020-05-22 Thread Jiamei Xie
Hi Kill all zookeeper and kafka process. Clear zookeeper and kafka data dir. Restart zookeeper and kafka. If there are any active client. Topic used by client will be auto-created. How to reproduce? 1. Start zookeeper and kafka zookeeper and kafka config file. nohup bin/zookeeper-server