Re: Priority queues using kafka topics and a consumer group?

2020-07-21 Thread Ricardo Ferreira
Richard, There is; but it doesn't look like what you would do in a typical messaging technology. As you may know Apache Kafka® is not messaging but a distributed streaming platform based on the concept of commit log. The main characteristic of a commit log is that they keep records in the sam

Priority queues using kafka topics and a consumer group?

2020-07-21 Thread Richard Ney
Trying to see if there is a way to implement priority topics where a single consumer group is reading from multiple topics each denoting a level of priority and having the Kafka client automatically prioritize reads against the message in the higher priority topics.

Out of range offset errors leading to offset reset

2020-07-21 Thread Varsha Abhinandan
Hi, We have been noticing offset resets happening on the kafka consumer because of offset out of range error. Following are the INFO logs found on the consumer side : [2020-07-17T08:46:00,322Z] [INFO ] [pipeline-thread-12 ([prd453-19-event-upsert]-bo-pipeline-12)] [o.a.k.c.consumer.internals.Fe