Thanks Matthias.
I have updated the KIP according to this and take KAFKA-6058.
We will use `KafkaConsumer` instead of reusing `ConsumerGroupCommand` and
keep `StreamsResetter` in `core` until `KAFKA-6058 is fixed.
Just as a reminder, [VOTE] thread is already open if there are no more
feedback on
Jorge,
thanks for the update.
I would suggest to not reuse `ConsumerGroupCommand` and re implement
what we need in `StreamsResetter` directly.
Even if we need to keep `StreamsResetter` in `core` for now, I think we
should not introduce new dependencies.
Currently, we still use old `kafka.admin.
Matthias,
Thanks for the heads up!
I think the main dependency is from `StreamResseter` to
`ConsumerGroupCommand` class to actually reuse `#reset-offsets`
functionality.
Not sure what would be the better way to remove it. To expose commands
(e.g. `ConsumerGroupCommand`) as part of AdminClient, t
Jorge,
KIP-198 (that got merged already) overlaps with this KIP. Can you please
update your KIP accordingly?
Also, while working on KIP-198, we identified some shortcomings in
AdminClient that do not allow us to move StreamsResetter our of core
package. We want to address those shortcoming in ano
Thanks Guozhang!
I have updated the KIP to:
1. Only one scenario param is allowed. If none, `to-earliest` will be used,
behaving as the current version.
2.
1. An exception will be printed mentioning that there is no existing
offsets registered.
2. inputTopics format could support define part
Hi Jorge,
Thanks for the KIP. It would be a great to add feature to the reset tools.
I made a pass over it and it looks good to me overall. I have a few
comments:
1. For all the scenarios, do we allow users to specify more than one
parameters? If not could you make that clear in the wiki, e.g. we
Thanks for the update Jorge.
I don't have any further comments.
-Matthias
On 8/12/17 6:43 PM, Jorge Esteban Quilcate Otoya wrote:
> I have updated the KIP:
>
> - Change execution parameters, using `--dry-run`
> - Reference KAFKA-4327
> - And advise about changes on `StreamResetter`
>
> Also i
I have updated the KIP:
- Change execution parameters, using `--dry-run`
- Reference KAFKA-4327
- And advise about changes on `StreamResetter`
Also includes that it will cover a change on `ConsumerGroupCommand` to
align execution options.
El dom., 16 jul. 2017 a las 5:37, Matthias J. Sax ()
escr
Thanks a lot for the update!
I like the KIP!
One more question about `--dry-run` vs `--execute`: While I agree that
we should use the same flag for both tools, I am not sure which one is
the better one... My personal take is, that I like `--dry-run` better.
Not sure what others think.
One more c
Hi,
KIP is updated.
Changes:
1. Approach discussed to keep both tools (streams application resetter and
consumer group reset offset).
2. Options has been aligned between both tools.
3. Zookeeper option from streams-application-resetted will be removed, and
replaced internally for Kafka AdminClient
Damian,
> resets everything and clears up
>> the state stores.
That's not correct. The reset tool does not touch local store. For this,
we have `KafkaStreams#clenup` -- otherwise, you would need to run the
tool in every machine you run an application instance.
With regard to state, the tool only
Hi,
Thanks for the KIP. What is not clear is how is this going to handle state
stores? Right now the streams reset tool, resets everything and clears up
the state stores. What are we going to do if we reset to a particular
offset? If we clear the state then we've lost any previously aggregated
val
You're right, I was not considering Zookeeper dependency.
I'm starting to like the idea to wrap `reset-offset` from
`streams-application-reset`.
I will wait a bit for more feedback and work on a draft with this changes.
El mié., 28 jun. 2017 a las 0:20, Matthias J. Sax ()
escribió:
> I agree,
I agree, that we should not duplicate functionality.
However, I am worried, that a non-streams users using the offset reset
tool might delete topics unintentionally (even if the changes are pretty
small). Also, currently the stream reset tool required Zookeeper while
the offset reset tool does not
Thanks for the feedback Matthias!
The main idea is to use only 1 tool to reset offsets and don't replicate
functionality between tools.
Reset Offset (KIP-122) tool not only reset but support execute the reset
but also export, import from files, etc.
If we extend the current tool (kafka-streams-app
Jorge,
thanks a lot for this KIP. Allowing the reset streams applications with
arbitrary start offset is something we got multiple requests already.
Couple of clarification question:
- why do you want to deprecate the current tool instead of extending
the current tool with the stuff the offset
Hi all,
I'd like to start the discussion to add reset offset tooling for Stream
applications.
The KIP can be found here:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-171+-+Extend+Consumer+Group+Reset+Offset+for+Stream+Application
Thanks,
Jorge.
17 matches
Mail list logo
