Error while deserializing the element

Setup Specifics: Version: 1.6.2 RocksDB Map State Timers stored in rocksdb When we have this job running for long periods of time like > 30 days, if for some reason the job restarts, we encounter "Error while deserializing the element". Is this a known issue fixed in later versions? I see some cha

Re: Loading Rules from compacted Kafka Topic - open() vs Connected Streams

functionality in flink Thanks. On Thu, Jul 12, 2018 at 1:00 PM vijayakumar palaniappan < vijayakuma...@gmail.com> wrote: > Hello All, > I can think of two options of implementing below requirement and request > some guidance on choosing the option with pros and cons. > > Requirem

Loading Rules from compacted Kafka Topic - open() vs Connected Streams

Hello All, I can think of two options of implementing below requirement and request some guidance on choosing the option with pros and cons. Requirements: - A in memory rules cache to be loaded from log compacted kafka topic. This cache has to be loaded prior to arrival of events. - Updates to th

Task Slots allocation

I am using flink 1.4.0 in standalone cluster mode I have a job with a graph like a Source(parallelism 3) -> Filter(parallelism 3) -> Map(parallelism 1) -> sink(parallelism 3) As per the understanding, max number of tasks slots should be bounded by the max parallelism, which in this case is 3. Th

Re: Incremental RocksDB checkpointing

al checkpoints and b) RocksDB at all. > > > Am 01.12.2017 um 16:34 schrieb vijayakumar palaniappan < > vijayakuma...@gmail.com>: > > > > I have simple event time window aggregate count function with > incremental checkpointing enabled. The checkpoint size keeps increasing &g

Incremental RocksDB checkpointing

I have simple event time window aggregate count function with incremental checkpointing enabled. The checkpoint size keeps increasing over a period of time, even though my input data has a single key and data is flowing at a constant rate. When i turn off incremental checkpointing, checkpoint size

REST Endpoint for Triggering Savepoint

Hi, Is there a REST endpoint for triggering savepoint without cancelling the job? -- Thanks, -Vijay

Windowed Stream Queryable State Support

What is the state of Windowed Stream Queryable State Support? Is it available in 1.3 or planned for 1.4? Thanks Vijay

CEP Pattern detection progressively slower

While trying to use Pattern detection on KeyedStream, list of unique keys encountered, keeps accumulating in AbstractKeyedCEPPatternOperator.java. On encountering watermarks these accumulated keys are looped over to detect patterns. This causes spike in CPU usage and progressively gets slower. Is

CEP timeout does not trigger under certain conditions

-TimeoutPattern does not trigger under certain conditions. Following are the preconditions: -Assume a pattern of Event A followed by Event B within 2 Seconds -PeriodicWaterMarks every 1 second -Assume follwoing events have arrived. -Event A-1[time: 1 sec] -Event B-1[time: 2 sec] -Event A-2[ti