Hi, Here is a use case that we would like to see kafka’s client support in the future. Currently reading a topic is FIFO. It would be awesome to read a topic in LIFO order. Put another way we would like to be able to read a topic in reverse.
Why? Basically we have per user streams which we have to keep up to date. When an event comes into the system it is determined to be cheap enough to fan out the user stream updates at write time or to expensive and thus deferred until read time. This is where kafka comes into play. We push all the events that were to expensive to fan out at write time onto a kafka topic. When a given user logs on we read through this topic and update the users stream. If it has been a while since the user has logged in it takes a while to catch up. If we could read a topic in reverse order we could catch the most recent events in the stream first. Which in our case is what the user cares about the most. Once the 20 most resent items have been added to a user's stream we can return the head of the stream and continue the catch up process in the background. Thanks, Jonathan
