Confluent Docker Images

The Docker images are huge for each confluent component - ZK, registry etc .. Is there any other place I can download one image that contains all the components ? REPOSITORY TAG IMAGE ID CREATED SIZE confluentinc/ksqldb-example

Confluent Platform- KTable clarification

I understood A KStream is an abstraction of a record stream and A KTable is an abstraction of a changelog stream ( updates or inserts) and the semantics around it. However, this is where some confusion arises .. From confluent documentation

Kafka - Controller Broker

come across this phrase from https://niqdev.github.io/devops/kafka/ and https://livebook.manning.com/book/kafka-streams-in-action/chapter-2/109 (Kafka Streams in Action ) The controller broker is responsible for setting up leader/follower relationships for all partitions of a topic. If a Kafka no

Confluent Kafka - Schema Registry on windows

I happened to see an example how to run schema registry using "schema-registry-start.bat" from windows on 5.0.1 I didnt see the file in 5.5.0 . Is the schema registry not supported in windows now ? IT seems only the way to go about running schema registry in windows through dockers . Please someon

Re: NotEnoughReplicasException: The size of the current ISR Set(2) is insufficient to satisfy the min.isr requirement of 3

ctor of > 3 so that one down or struggling broker doesn't prevent producers writing > to topics, but I still have a replica of the data in case the broker acting > as leader goes down - a new partition leader can only be elected from the > insync replicas. > > On Tue, Jul

NotEnoughReplicasException: The size of the current ISR Set(2) is insufficient to satisfy the min.isr requirement of 3

I had the following setup Brokers : 3 - all are up and running with min.insync.replicas=3. I created a topic with the following configuration bin\windows\kafka-topics --zookeeper 127.0.0.1:2181 --topic topic-ack-all --create --partitions 4 --replication-factor 3 I triggered the producer with "ac

Re: Highwater mark interpretation

ess costs). > > But, I've never encountered a situation where an underreplicated partition > prevents consuming. > > Cheers, > > Liam Clarke-Hutchinson > > On Sun, 21 Jun. 2020, 2:08 pm Nag Y, wrote: > > > Thanks D C. Thanks a lot . That is quite a detaile

Re: Highwater mark interpretation

ay be lower than the High Watermark so if your consumer is accessing > the data in Read_Committed, it won't be able to surpass the LSO. > > Cheers, > D > > On Sat, Jun 20, 2020 at 9:05 PM Nag Y wrote: > > > As I understand it, the consumer can only read "committ

Highwater mark interpretation

As I understand it, the consumer can only read "committed" messages - which I believe, if we look at internals of it, committed messages are nothing but messages which are upto the high watermark. *The high watermark is the offset of the last message that was successfully copied to all of the log’s

Re: Uneven distribution of messages in topic's partitions

Hi Ricardo , Just follow up question to add , I believe the defaultpartioner uses mumur3 as default . Should RoundRobinPartitioner class be used to have an equal distribution to maximum extent.instead of default partitioner ? Is StickyPartitioner (mentioned above) is different from Roun

Kafka - Failed to clean up log for __consumer_offsets-10 in dir

I am seeing the following exception in one of the broker log files. Set up contains 3 brokers. Environment - Windows I am ok to remove the files c:\tmp directory. However, I'm a little curious to know why this broker got into this state and if there is a way to rectify the issue without deleting

Re: Kafka - replicas do not heal themselves by default

use > cases > > Could you share more details for the actual scenarios you are working on > outside the book? > > > > On Sun, Jun 14, 2020 at 1:56 PM Nag Y wrote: > > > I am going through the kafka in action and come across this following > > phrase > >

Kafka - replicas do not heal themselves by default

I am going through the kafka in action and come across this following phrase *One of the things to note with Kafka is that replicas do not heal themselves by default. If you lose a broker on which one of your copies of a partition existed, Kafka does not currently create a new copy. I mention this

Re: Kafka - how to know whether it is broker property or topic property or producer property

is given the > server default is used. The override can be set at topic creation time by > giving one or more --config options. > > > Hope this helps :) > > Cheers, > > Liam Clarke-Hutchinson > > > On Sun, Jun 14, 2020 at 6:48 PM Nag Y wrote: > > > Is it doc

Re: Kafka - how to know whether it is broker property or topic property or producer property

t; that the broker property is the default, which can be overridden by > explicit topic configs. > > Kind regards, > > Liam Clarke > > On Sun, Jun 14, 2020 at 6:42 PM Nag Y wrote: > > > I am going through the documentation and often times, it is either not > >

Kafka - how to know whether it is broker property or topic property or producer property

I am going through the documentation and often times, it is either not clear or need to look at in multiple pleaces to see to which a prticular property belongs and is it a specific property to an entity etc .. To give an example, consider *"min.insync.replicas"* - This is just for an example. Fro

Re: Kafka - entity-default Vs entity-name

will show you overridden --entity-defaults > (DYNAMIC_DEFAULT_BROKER_CONFIG) and --entity-name 1 > (DYNAMIC_BROKER_CONFIG). > > Brian > > On Fri, Jun 12, 2020 at 8:12 AM Nag Y wrote: > > > Thanks Brian. I applied the following command with --all, it works fine. > > > > k

Re: Kafka - entity-default Vs entity-name

to see > all config entries, you can pass flag --all which is included in the 2.5 > release (KIP-524: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-524%3A+Allow+users+to+choose+config+source+when+describing+configs > ). > > Please let me know if you have any other ques

Re: Kafka - entity-default Vs entity-name

t; > https://stackoverflow.com/questions/62343035/kafka-entity-default-vs-entity-name > > > -- > > Robin Moffatt | Senior Developer Advocate | ro...@confluent.io | @rmoff > > > On Fri, 12 Jun 2020 at 13:13, Nag Y wrote: > > > I applied kafka-config to get the default

Kafka - entity-default Vs entity-name

I applied kafka-config to get the default settings for the brokers, kafka-configs --bootstrap-server localhost:9092 --entity-type brokers --entity-default --describe The command responded with the following response, without any *complete* output. Default configs for brokers in the cluster are:

Kafka - Auto commit

During the time when i was going through the doc and come across , Automatic Commit The easiest way to commit offsets is to allow the consumer to do

Kafka - entity-default Vs entity-name

I applied kafka-config to get the default settings for the brokers, kafka-configs --bootstrap-server localhost:9092 --entity-type brokers --entity-default --describe The command responded with the following response, without any *complete* output. Default configs for brokers in the cluster are: