Sophie,
In continuation to the discussion above, now that we are using redis for
storing state in our application which breaks the EOS of kafka streams, I
do have one question and whether we can do some work around here: in the
scenario :
1. Event consumed from source
2. Event processed and state
>
> it seems to me that local state stores might prove more performant in the
> event of above
> mentioned failures of streams application
Note, the scenario I was describing with deleting all record from the
remote store in event of
failure is *not* what we do today -- I was describing a potentia
Thanks Sophie... i was just thinking what would be a good options for us,
whether local state stores or redis state store, and it seems to me that
local state stores might prove more performant in the event of above
mentioned failures of streams application.
The main reason we are thinking of movi
Yes, if you have to restore from the changelog from scratch then this will
definitely impact
the application's performance. This is the current state of things for EOS
applications that use
some kind of local storage such as the in-memory or rocksdb state
stores.The point of EOS is
to be 100% corre
Thanks Sophie... that answers my question, however still worried about some
other aspects:
1. If redis is to be restored from changelog topic: what would happen if i
have 3 stream applications and 1 instance went down ... will other 2
instances halt until entire existing state from redis is wiped
Hey Pushkar, yes, the data will still be backed by a changelog topic unless
the
user explicitly disables logging for that state store. The fault tolerance
mechanism
of Kafka Streams is based on changelogging, therefore there are no
correctness
guarantees if you decide to disable it.
That said, I'm
Hello Sophie,
may be i am missing something here, however can you let me know how a redis
based state store will be wiped off the inconsistent state in case stream
application dies in the middle of processing e.g. stream application
consumed from source topic, processed source event and saved stat
Thanks!
On Mon, Mar 15, 2021 at 8:38 PM Sophie Blee-Goldman
wrote:
> Yep, that fell off my radar. Here we go:
> https://issues.apache.org/jira/browse/KAFKA-12475
>
> On Mon, Mar 15, 2021 at 8:09 PM Guozhang Wang wrote:
>
> > Hey Sophie,
> >
> > Maybe we can first create a JIRA ticket for this?
Yep, that fell off my radar. Here we go:
https://issues.apache.org/jira/browse/KAFKA-12475
On Mon, Mar 15, 2021 at 8:09 PM Guozhang Wang wrote:
> Hey Sophie,
>
> Maybe we can first create a JIRA ticket for this?
>
> On Mon, Mar 15, 2021 at 3:09 PM Sophie Blee-Goldman
> wrote:
>
> > Sounds good
Hey Sophie,
Maybe we can first create a JIRA ticket for this?
On Mon, Mar 15, 2021 at 3:09 PM Sophie Blee-Goldman
wrote:
> Sounds good! I meant anyone who is interested :)
>
> Let me know if you have any questions after digging in to this
>
> On Mon, Mar 15, 2021 at 2:39 PM Alex Craig wrote:
>
Sounds good! I meant anyone who is interested :)
Let me know if you have any questions after digging in to this
On Mon, Mar 15, 2021 at 2:39 PM Alex Craig wrote:
> Hey Sophie, not sure if you meant me or not but I'd be happy to take a
> stab at creating a KIP for this. I want to spend some ti
Hey Sophie, not sure if you meant me or not but I'd be happy to take a
stab at creating a KIP for this. I want to spend some time digging into
more of how this works first, but will then try to gather my thoughts and
get something created.
Alex
On Mon, Mar 15, 2021 at 1:48 PM Sophie Blee-Goldma
This certainly does seem like a flaw in the Streams API, although of course
Streams is just
in general not designed for use with remote anything (API calls, stores,
etc)
That said, I don't see any reason why we *couldn't* have better support for
remote state stores.
Note that there's currently no
Hi Alex,
I guess wiping out the state directory is easier code-wise, faster,
and/or at the time of development the developers did not design for
remote state stores. But I do actually not know the exact reason.
Off the top of my head, I do not know how to solve this for remote state
stores.
Bruno,
Thanks for the info! that makes sense. Of course now I have more
questions. :) Do you know why this is being done outside of the state
store API? I assume there are reasons why a "deleteAll()" type of function
wouldn't work, thereby allowing a state store to purge itself? And maybe
mor
Bruno,
i tried to explain this in 'kafka user's language through above mentioned
scenario, hope i put it properly -:) and correct me if i am wrong
On Mon, Mar 15, 2021 at 7:23 PM Pushkar Deole wrote:
> This is what I understand could be the issue with external state store:
>
> kafka stream appl
This is what I understand could be the issue with external state store:
kafka stream application consumes source topic, does processing, stores
state to kafka state store (this is backed by topic) and before producing
event on destination topic, the application fails with some issue. In
this case,
Hi Alex,
You are right! There is no "exactly once magic" backed into the RocksDB
store code. The point is local vs remote. When a Kafka Streams client
closes dirty under EOS, the state (i.e., the content of the state store)
needs to be wiped out and to be re-created from scratch from the
chan
" Another issue with 3rd party state stores could be violation of
exactly-once guarantee provided by kafka streams in the event of a failure
of streams application instance"
I've heard this before but would love to know more about how a custom state
store would be at any greater risk than RocksDB
Thanks for the responses. In the worst case, I might have to keep both rocksdb
for local store and keep an external store like Redis.
-mohan
On 3/13/21, 8:53 PM, "Pushkar Deole" wrote:
Another issue with 3rd party state stores could be violation of
exactly-once guarantee provided by
Another issue with 3rd party state stores could be violation of
exactly-once guarantee provided by kafka streams in the event of a failure
of streams application instance.
Kafka provides exactly once guarantee for consumer -> process -> produce
through transactions and with the use of state store l
Hello Mohan,
I think what you had in mind works with Redis, since it is a remote state
store engine, it does not have the co-partitioning requirements as local
state stores.
One thing you'd need to tune KS though is that with remote stores, the
processing latency may be larger, and since Kafka St
Hi,
I have a use case where messages come in with some key gets assigned some
partition and the state gets created. Later, key changes (but still contains
the old key in the message) and gets sent to a different partition. I want to
be able to grab the old state using the old key before creatin
23 matches
Mail list logo
