> same time I would prefer additional churn to the API.
I meant to say prefer _less_ churn to the API.
On Tue, Jun 09, 2015 at 08:35:35AM -0700, Joel Koshy wrote:
> Ewen,
>
> Sorry for the late comment, but while we are discussing this API I
> think we should also consider the addition of metada
Ewen,
Sorry for the late comment, but while we are discussing this API I
think we should also consider the addition of metadata to the offset
commit. This is supported on the broker-side but there is no means in
the current API to include metadata or retention time in the offset
commit. Ideally,
Thanks Ewen,
1. I agree with you as for the pros of Future; my argument for pure
Callback, as I mentioned, is that it sounds to me unlikely users would
usually want to explicitly set timeout. That said, if people think this
scenario is actually common like in graceful shutdown, then I am OK with
F
Thanks, great feedback everyone.
Jiangjie -- I was worried about interleaving as well. For commits using the
consumer's own current set of offsets, I agree we could easily limit to 1
outstanding request so the older one gets cancelled. For commits that
specify offsets manually, we might not need t
Hi Ewen,
Only time I can think of where Application needs to know result of offset
was committed or not during graceful shutdown and/or
Runtime.addShutdownHook() so consumer application does not get duplicated
records upon restart or does not have to deal with eliminating already
process offset.
I second Guozhang's proposal. I do think we need the callback. The current
state is that for async commits you actually don't know if it succeeded.
However there is a totally valid case where you do need to know if it
succeeded but don't need to block, and without the callback you are stuck.
I thin
Hi Ewen,
I share the same concern you have about 2), that with the new API sync
commit implementation is a bit awkward since we have a single-threaded
design in new consumer. The reason that we need to mute other nodes for
doing coordinator sync operations like join-group / offset commits / offset
Hey Ewen,
This makes sense. People usually do not want to stop consuming when
committing offsets.
One corner case about async commit with retries I am thinking is that it
is possible that two offset commits interleave with each other and that
might create problem. Like you said maybe we can canc
