hi Roger, What consumer u r using? Is there a chance to mention consumer threads?
Example: logstash kafka consumer has configurable number of threads under each consumer instance. That may help up to some extent. Regards, Sunil. On Thu, 6 Jan 2022 at 7:27 PM, Roger Kasinsky <roger.kasin...@gmail.com> wrote: > Hi Luke, > > > The solution I can think of is to create only one partition for the > topic. > > That would work, but then I lose the benefits of the partitions. > > > Or you can create 4 consumers in one group, to consume from 4 partitions. > That works, too. > > That does not work, because I need only one consumer receiving all the > messages in the same order on every run. > > Hi Suni, > > > Why dont you provide new name to consumer group each time you restart > your consumer? This new consumer group will not conflict with the earlier > one and it will be treated as new consumer thread next time to get all > messages again. > > That does not solve my problem, which is to have the same consumer getting > all the topic messages in the same order. I'm not worried about conflicts. > In fact, I want the exact same consumer to run twice in a row. Renaming the > consumer group does not help with anything related to message order. > > Thanks! > > -R > > > > On Thu, Jan 6, 2022 at 12:21 AM sunil chaudhari < > sunilmchaudhar...@gmail.com> > wrote: > > > hi, > > Why dont you provide new name to consumer group each time you restart > your > > consumer? > > This new consumer group will not conflict with the earlier one and it > will > > be treated as new consumer thread next time to get all messages again. > > > > > > Regards, > > Sunil. > > > > On Wed, 5 Jan 2022 at 10:45 PM, Roger Kasinsky <roger.kasin...@gmail.com > > > > wrote: > > > > > Hi, > > > > > > I have a topic divided into 4 partitions. I have a consumer that needs > to > > > consume all messages from the topic (all messages from all 4 > partitions). > > > So to do that I have this consumer sitting by itself in its own > consumer > > > group. I'm not committing any offsets, because I want to read all > > messages > > > again on every restart of the consumer. > > > > > > *Question:* *Can anything be said about the order of the messages > > consumed > > > by my consumer? Is there a way to enforce the same order of messages > for > > > every restart of my consumer?* > > > > > > Thanks! > > > > > > -R > > > > > >
