> On Oct 9, 2017, at 2:41 PM, John Walker <johnwalk...@gmail.com> wrote: > > I have a pair of services. One dispatches commands to the other for > processing. > > My consumer sometimes fails to execute commands as a result of transient > errors. To deal with this, commands are retried after an exponentially > increasing delay up to a maximum of 4 times. (Delays: 1hr, 2hr, 4hr, 8hr.) > What's the standard way to set up something like this using Kafka? > > The only solution I've found so far is to setup 5 topics (main_topic, > delayed_1hr, delayed_2hr, delayed_4hr,delayed_8hr), and then have pollers > that poll each of these topics, enforce delays, and escalate messages from > one topic to another if errors occur.
Our approach for a similar scheduling problem was to assign each command a unique key and a "valid after" date. Any time you fail to execute a command, you update the "valid after" with your exponential backoff algorithm, and produce the updated value over the same key. Old versions are removed by compaction. Biggest downside is now your queue is no longer ordered, but for our problem in practice the number of pending messages is relatively small so we simply scan all outstanding messages every delivery interval to see if any is eligible for a retry.
signature.asc
Description: Message signed with OpenPGP using GPGMail
