Reducing issue

2022-02-01 Thread Robin Helgelin
Hi, Working on a small MVP and keep running into a dead end when it comes to reducing data. Began using ksql, which worked very well to start with. Basically, I have two streams imported via a HTTP source connector. These streams are then merged to a single stream and some extra information fr

Re: Kafka Consumer Fairness when fetching events from different partitions.

2022-02-01 Thread Mazen Ezzeddine
Hello Chen and Edward , I have went as well through the documentation in https://cwiki.apache.org/confluence/display/KAFKA/KIP-41%3A+KafkaConsumer+Max+Records#KIP41:KafkaConsumerMaxRecords-EnsuringFairConsumption And it turns out that the consumer polls in a greedy round-robin algorithm. Confi

ReplicaManager OutOfOrderSequenceException

2022-02-01 Thread Nicolas Carlot
Hello, I'm having multiple errors in our kafka cluster server.log regarding a single topic and a subset of its partitions. ERROR [ReplicaManager broker=1] Error processing append operation on partition declaration.amount-to-c50builder-30 (kafka.server.ReplicaManager) org.apache.kafka.common.error

Can you use the log4j 2 bridge to replace log4j 1.x in Kafka?

2022-02-01 Thread Steve Souza
Where I work we use both Tableau and Microstrategy which in turn use Kafka. Due to recent log4j vulnerabilities security is coming down on any product using the End-Of-Life log4j 1.x. Log4j 1.x was end-of-lifed in 2015. Kafka uses log4j 1.x. Security is no longer accepting a vendor statement tha