7;s state size discrepancy with what's seen with state
processor API
Ok
Regards,
Alexis.
From: Peter Brucia
Sent: Freitag, 22. April 2022 15:31
To: Alexis Sarda-Espinosa
Subject: Re: RocksDB's state size discrepancy with what's seen with state
processor API
No
Sent from my iPhone
Ok
Regards,
Alexis.
From: Peter Brucia
Sent: Freitag, 22. April 2022 15:31
To: Alexis Sarda-Espinosa
Subject: Re: RocksDB's state size discrepancy with what's seen with state
processor API
No
Sent from my iPhone
2, 2022 9:57 AM
To: Alexis Sarda-Espinosa
Cc: ro...@apache.org ; user@flink.apache.org
Subject: Re: RocksDB's state size discrepancy with what's seen with state
processor API
Alexis,
Compaction isn't an all-at-once procedure. RocksDB is organized as a series of
levels, e
.
>
> Regards,
> Alexis.
>
> -Original Message-
> From: Roman Khachatryan
> Sent: Mittwoch, 20. April 2022 10:37
> To: Alexis Sarda-Espinosa
> Cc: user@flink.apache.org
> Subject: Re: RocksDB's state size discrepancy with what's seen with state
> p
Sarda-Espinosa
Cc: user@flink.apache.org
Subject: Re: RocksDB's state size discrepancy with what's seen with state
processor API
State Processor API works on a higher level and is not aware of any RocksDB
specifics (in fact, it can be used with any backend).
Regards,
Roman
On Tue, Apr 1
___
> From: Roman Khachatryan
> Sent: Tuesday, April 19, 2022 5:51 PM
> To: Alexis Sarda-Espinosa
> Cc: user@flink.apache.org
> Subject: Re: RocksDB's state size discrepancy with what's seen with state
> processor API
>
> > I assume that when
ts?
Regards,
Alexis.
From: Roman Khachatryan
Sent: Tuesday, April 19, 2022 5:51 PM
To: Alexis Sarda-Espinosa
Cc: user@flink.apache.org
Subject: Re: RocksDB's state size discrepancy with what's seen with state
processor API
> I assume that when you say &qu
05
>
> Regards,
> Alexis.
>
> -----Original Message-
> From: Roman Khachatryan
> Sent: Dienstag, 19. April 2022 11:48
> To: Alexis Sarda-Espinosa
> Cc: user@flink.apache.org
> Subject: Re: RocksDB's state size discrepancy with what's seen with state
> proce
lexis Sarda-Espinosa
Cc: user@flink.apache.org
Subject: Re: RocksDB's state size discrepancy with what's seen with state
processor API
Hi Alexis,
Thanks a lot for the information,
MANIFEST files list RocksDB column families (among other info); ever growing
size of these files might indi
growing, maybe I
> > can rule out expired state staying around?. My pipeline doesn't use timers,
> > although I guess Flink itself may use them. Is there any way I could get
> > some insight into which operator holds larger states?
> >
> > Regards,
> > A
osa
Sent: Dienstag, 12. April 2022 15:39
To: ro...@apache.org
Cc: user@flink.apache.org
Subject: RE: RocksDB's state size discrepancy with what's seen with state
processor API
Thanks for all the pointers. The UI does show combined state for a chain, but
the only state descriptors inside th
rs save some information in the state as well.
Regards,
Alexis.
-Original Message-
From: Roman Khachatryan
Sent: Dienstag, 12. April 2022 14:06
To: Alexis Sarda-Espinosa
Cc: user@flink.apache.org
Subject: Re: RocksDB's state size discrepancy with what's seen with state
proces
use timers,
> although I guess Flink itself may use them. Is there any way I could get some
> insight into which operator holds larger states?
>
> Regards,
> Alexis.
>
> -Original Message-----
> From: Roman Khachatryan
> Sent: Dienstag, 12. April 2022 12:37
> To: Alexi
s,
Alexis.
-Original Message-
From: Roman Khachatryan
Sent: Dienstag, 12. April 2022 12:37
To: Alexis Sarda-Espinosa
Cc: user@flink.apache.org
Subject: Re: RocksDB's state size discrepancy with what's seen with state
processor API
Hi Alexis,
Thanks a lot for sharing this. I t
t?
>
>
>
> Regards,
>
> Alexis.
>
>
>
> From: Alexis Sarda-Espinosa
> Sent: Samstag, 9. April 2022 01:39
> To: ro...@apache.org
> Cc: user@flink.apache.org
> Subject: Re: RocksDB's state size discrepancy with what's seen with state
>
ds,
Alexis.
From: Alexis Sarda-Espinosa
Sent: Samstag, 9. April 2022 01:39
To: ro...@apache.org
Cc: user@flink.apache.org
Subject: Re: RocksDB's state size discrepancy with what's seen with state
processor API
Hi Roman,
Here's an example of a WindowReaderFunction:
with
parallelism=4, this doesn't affect the result, does it?
Regards,
Alexis.
From: Roman Khachatryan
Sent: Friday, April 8, 2022 11:06 PM
To: Alexis Sarda-Espinosa
Cc: user@flink.apache.org
Subject: Re: RocksDB's state size discrepancy with what's
Hi Alexis,
If I understand correctly, the provided StateProcessor program gives
you the number of stream elements per operator. However, you mentioned
that these operators have collection-type states (ListState and
MapState). That means that per one entry there can be an arbitrary
number of state
Hello,
I have a streaming job running on Flink 1.14.4 that uses managed state with
RocksDB with incremental checkpoints as backend. I've been monitoring a dev
environment that has been running for the last week and I noticed that state
size and end-to-end duration have been increasing steadily.
19 matches
Mail list logo
