Hi Naveen, Is your deferred processing logic respect ordering still or not? More specifically let's say if you published records A, B, C, D ... to the same topic partition and let's say record B's processing need to be delayed, then would 1) records C and D and all following are okay to be deferred by the same amount of time so that they are still only processed after record B, or 2) some later records in C, D, ... still need to be processed immediately?
In case 2) there's not a very good solution so far since Kafka generally need to follow offset ordering in fetching / processing, maybe you can consider reading the deferred records immediately and then re-send it to another "deferred topic", which would only be read after some time while normally processing other records; In case 1) there are several options you can consider, e.g. once after you've fetched record B and realized you need to enter the "deferred processing" phase (so that this record and all future records should be deferred), you can pause the corresponding partition in your consumer will just hold on the record B at hand, continue processing other partitions until the deferred time has elapsed. Hope those helps. Guozhang On Thu, Oct 31, 2019 at 10:00 PM Naveen kumar mandavilli < naveenkumar....@gmail.com> wrote: > Hi > > We have a use case where we have to defer processing the messages by one > hour from the time it got published. Please suggest if there is an elegant > solution to handle this. > > Thanks > Naveen kumar M. > -- > Naveen kumar M. > -- -- Guozhang
