Sorry for the late reply.
> You mentioned that, reduce() could use RocksDB as stores by default
> while suppress() is in memory. Is that the reason that reduce() has
> both -repartition and -changelog topics while suppress() only has
> -changelog topic?
No. The repartitioning would happen if you
Hi Matthias,
Could you help with above issue? Or any suggestions?
Thanks a lot!
On Thu, Oct 31, 2019 at 4:00 PM Xiyuan Hu wrote:
>
> Hi Matthias,
>
> Some additional information, after I restart the app, it went to
> endless rebalancing. Join rate loos like below attachment. It's
> basically re
Hi Matthias,
Some additional information, after I restart the app, it went to
endless rebalancing. Join rate loos like below attachment. It's
basically rebalanced every 5 minutes. I checked into each node
logging. And found below warning:
On node A:
2019/10/31 10:13:46 | 2019-10-31 10:13:46,543 W
Hi Matthias,
When I redeployment the application with the same application Id, it
will cause a rebalance loop: partition revoked -> rebalance -> offset
reset to zero -> partition assigned -> partition revoked.
The app was running well before the redeployment, but once redeployed,
it will keep reb
Just a follow up: currently, suppress() only supports in-memory stores
(note, that `suppress()` has it's own store).
For the actually `reduce()` store, you can pick between RocksDB and
in-memory (default is RocksDB). Hence, if you restart an application on
the same host, it should not be necessary
Wang
Sent: Tuesday, October 29, 2019 10:09 AM
To: users@kafka.apache.org
Subject: RE: kafka stream ktable with suppress operator
Thanks Matthia for the reply. I am using kafka-stream-2.1.0 jar. I will try
2.3.1 and see if the issue still there.
Tao
-Original Message-
From: Matthias J
ktable with suppress operator
What version are you using? We fixed couple of bugs in `suppress()` -- I
would recommend to use latest 2.3.1 bug-fix release.
-Matthia
On 10/25/19 9:12 AM, Tao Wang wrote:
> When using suppress operator with windowed Ktable, it looks like restarting
> the
I assume you're using RocksDB as your state stores... The bytes out you're
seeing on the changelog topics is probably because they are restoring your
state stores. If your state stores are in-memory, then on every
application startup they're going to be restored from the changelog
topics. If your
Hi,
I'm using 2.3.1 now and having the same issue. During restarting, I
noticed a lot logging like below:
Seeking to EARLIEST offset of partition
XX-KSTREAM-REDUCE-STATE-STORE-14-changelog-41
Seeking to EARLIEST offset of partition
XX-KTABLE-SUPPRESS-STATE-STORE-20-changelog-41
Aft
When using suppress operator with windowed Ktable, it looks like restarting the
kafka stream causes the aggregated messages from the SUPPRESS-STATE-STORE
published again..
Here is the sudo code .. anything I am missing or anything can be done to avoid
this ..
KTable, String> test =
.groupByK
10 matches
Mail list logo
