Alright. Here is an approach that I can think of. Create a topic for each delay and produce message to the appropriate topic. Each produced message should have a timestamp at which it was pushed to the queue. At the consumer side, fetch a message from a partition and compare the message timestamp with system's timestamp to see if enough time has passed for you to process the message. If enough time has passed, process the message and commit the message's offset otherwise make sure you do not commit the offset.
You may have to use low level consumer API to achieve this, however double check if you can use high level consumer API to achieve the same (as I am not an expert of the consumer APIs). Also, this assumes the size of your delay set is small as it is recommended to only have couple of thousand partitions. On 23 Feb 2014 15:15, "Jagan Ranganathan" <ja...@zohocorp.com> wrote: > Hi Aniket, > > > I am looking at "Finite set of delays" and messages will use one on the > defined set. > > > Regards, > Jagan > > ---- On Sun, 23 Feb 2014 15:13:02 +0530 Aniket > Bhatnagar<aniket.bhatna...@gmail.com> wrote ---- > > > Hi Jagan > > Are you expecting each message to have the same delay? Or is there a finite > set of delays? Or messages can arbitrarily any delay? > > Thanks, > Aniket > On 23 Feb 2014 14:13, "Jagan Ranganathan" <ja...@zohocorp.com> > wrote: > > > Dear All, > > > > I am looking for a Delayed Queue implementation for my distributed > system > > requirement. Would like to know if I can achieve the same using Kafka. > > > > > > Regards, > > Jagan > > > > > > > >
