Kafka - entity-default Vs entity-name

I applied kafka-config to get the default settings for the brokers, kafka-configs --bootstrap-server localhost:9092 --entity-type brokers --entity-default --describe The command responded with the following response, without any *complete* output. Default configs for brokers in the cluster are:

Kafka - Auto commit

During the time when i was going through the doc and come across , Automatic Commit The easiest way to commit offsets is to allow the consumer to do

Kafka - entity-default Vs entity-name

I applied kafka-config to get the default settings for the brokers, kafka-configs --bootstrap-server localhost:9092 --entity-type brokers --entity-default --describe The command responded with the following response, without any *complete* output. Default configs for brokers in the cluster are:

MM2 configuration

Hi, i am trying to migrate from mm1 to mm2 i am trying to run: ./bin/connect-mirror-maker.sh ./config/mirror-maker.properties i have following configuration: clusters = A, B connector.client.config.override.policy = All A.bootstrap.servers = hostA1:9093, hostA2:9093 B.bootstrap.servers = hostB

Re: Kafka - entity-default Vs entity-name

If you're going to cross-post, it's useful to include where else you've asked this before, so that people can see what else has been suggested and answered to avoid duplicating their effort :) https://stackoverflow.com/questions/62343035/kafka-entity-default-vs-entity-name -- Robin Moffatt | S

Re: Kafka - entity-default Vs entity-name

sure On Fri, Jun 12, 2020 at 6:18 PM Robin Moffatt wrote: > If you're going to cross-post, it's useful to include where else you've > asked this before, so that people can see what else has been suggested and > answered to avoid duplicating their effort :) > > > https://stackoverflow.com/questio

Re: Kafka Connect Connector Tasks Uneven Division

Hi Robin Request you to please reply. Regards and Thanks Deepak Raghav On Wed, Jun 10, 2020 at 11:57 AM Deepak Raghav wrote: > Hi Robin > > Can you please reply. > > I just want to add one more thing, that yesterday I tried with > connect.protocal=eager. Task distribution was balanced after

Re: Kafka - entity-default Vs entity-name

Hi Nag, To address (2) first, the --entity-default flag requests the default configuration that all brokers inherit. An individual broker may override any of the default config's entries, which is done by specifying the broker ID to the --entity-name flag. The reason you're getting blank output f

kafka-mirror 2 configuration

Hi, i am trying to migrate from mm1 to mm2 i am trying to run: ./bin/connect-mirror-maker.sh ./config/mirror-maker.properties i have following configuration: clusters = A, B connector.client.config.override.policy = All A.bootstrap.servers = hostA1:9093, hostA2:9093 B.bootstrap.servers = hostB

Re: Kafka - entity-default Vs entity-name

Thanks Brian. I applied the following command with --all, it works fine. kafka-configs --bootstrap-server localhost:9092 --entity-type brokers --entity-name 1 --describe --all. I believe this includes *both* default config values as well as config values those were overridden . Is there any com

Re: Kafka - entity-default Vs entity-name

Hi Nag, Correct, --all will include both. Removing --all should give you only the overridden values. Are you asking if you can see overridden values for a broker with both the default config and specific broker config in one command? Unfortunately the commands only give you the overrides for that

Re: Kafka - entity-default Vs entity-name

My apologies, the command mentioned should include --all: kafka-configs ... --entity-type brokers --entity-name 1 --describe --all | grep DYNAMIC Brian On Fri, Jun 12, 2020 at 8:45 AM Brian Byrne wrote: > Hi Nag, > > Correct, --all will include both. Removing --all should give you only the >

dynamic produce and consume new topics

I'm working on a data streaming system where users will publish to different new topics and subscribe to different new topics without restarting the system. A large number of topics will be created over time. Uer will choose different new topics to publish and subscribe dynamically at runtime. Can

Re: dynamic produce and consume new topics

Hey AJ, you should be able to make the subscribed topics and output topics dynamic configs for Consumer and Producer. If you need to create new topics on runtime, there is some broker side setting you could use to allow topic creation based of produced records: https://stackoverflow.com/questions/

Re: Kafka - entity-default Vs entity-name

Thanks Brian, that answers my question On Fri, 12 Jun, 2020, 9:16 PM Brian Byrne, wrote: > Hi Nag, > > Correct, --all will include both. Removing --all should give you only the > overridden values. > > Are you asking if you can see overridden values for a broker with both the > default config an

Basic Auth validation for Kafka Connect REST Interface - credentials caching

Hi Team, I have implemented Basic auth validation for Kafka Connect REST Interface. I noticed that the worker process needs the password file only till the time first query is made to the REST Interface. Post the first query even if we delete the file containing password , we can keep querying the

Re: Using encrypted distributed worker file

Hi Team, Any help would be greatly appreciated. I am looking forward to a way where I can store the passwords in encrypted/hashed format on the distributed worker properties file. Regards Ashish Sood On Tue, Jun 2, 2020 at 12:42 AM ashish sood wrote: > Hi All, > > I am running a distributed w

Re: dynamic produce and consume new topics

Thanks, Boyang. Spring''s KafkaTemplate can easily create new topic on the flight already. On consumer side, is there java sample code to replace this annotation? That will create a new listener for a new topic and group-id on the fliight. @KafkaListener(topics = "test", groupId = "grp0") -aj

request-reply model

Hi anyone has instructions for setting up request-reply messaging using kafka? thanks

Re: dynamic produce and consume new topics

I figured out this approach to consume new topics that become available at runtime: 1. create ConcurrentMessageListenerContainer for each new topic, call start() to start listening. 2. track all containers in a map, which listen and consume messages from the assigned topics. 3. call container.stop(

Re: request-reply model

Roman, If you are implementing this in Java and use the Spring Framework then there is a good support in there that abstracts away much of the complexity related to correlate requests and responses. Here is an article that gives an example of how to implement this: https://asbnotebook.com/20