Sure. Sorry I was not clear.
Thank you!
lör 10 mars 2018 kl. 00:54 skrev Matthias J. Sax :
> If there is only one partition by task, processing order is guaranteed.
>
> For default partitions grouper, it depends on your program. If you read
> from multiple topics and join/merge them, a task get
If there is only one partition by task, processing order is guaranteed.
For default partitions grouper, it depends on your program. If you read
from multiple topics and join/merge them, a task gets multiple
partitions (from different topics) assigned.
-Matthias
On 3/9/18 2:42 PM, Stas Chizhov w
> Also note, that the processing order might slightly differ if you
process the same data twice
Is this still a problem when default partition grouper is used (with 1
partition per task)?
Thank you,
Stanislav.
2018-03-09 3:19 GMT+01:00 Matthias J. Sax :
> Thanks for the explanation.
>
>
Thanks for the explanation.
Not sure if setting the metadata you want to get committed in
punctuation() would be sufficient. But I would think about it in more
details if we get a KIP for this.
It's correct that flushing and committing offsets is correlated. But
it's not related to punctuation.
Thank you, Matthias!
We currently do use kafka consumer and store event time highwatermarks as
offset metadata. This is used during backup procedure, which is to create a
snapshot of the target storage with all events up to certain timestamp and
no other.
As for the API - I guess being able to pr
You are correct. This is not possible atm.
Note, that commits happen "under the hood" and users cannot commit
explicitly. Users can only "request" as commit -- this implies that
Kafka Streams will commit as soon as possible -- but when
`context#commit()` returns, the commit is not done yet (it onl
Hi,
There seems to be no way to commit custom metadata along with offsets from
within Kafka Streams.
Are there any plans to expose this functionality or have I missed something?
Best regards,
Stanislav.
