> On Oct 26, 2018, at 2:00 PM, Colin McCabe <cmcc...@apache.org> wrote:
>>
> Priorities won't help for this use-case, right? If the "web" partition has a
> higher priority, and data is always available, there will *never* be any
> events reported for "sync". Priorities don't prevent starvation-- they cause
> starvation by design, because the high priority partition always takes
> priority.
Starvation is certainly an issue, though we could include a timeout as Bala
suggested to address this.
> In general the best solution would probably be to have a work queue between
> the consumer and the event handler, and manage the backpressure as
> appropriate. This could be done with pause and resume, as Streams does.
I agree that similar semantics could be achieved with a work queue. What we’re
voting on is the merits topic prioritization to make the API more expressive
and to make it easier for developers to do this.
Thanks Colin for your vote on the KIP and for all you input. I look forward to
hearing from others.
Cheers,
--
Nick
