Thanks for your reply.
It's a streaming job. The join operator is doing join work, such as join.
The join state is too large so I don't want to keep the state using the
mechanism that Flink provided, and also I don't need very precise join. So
I prefer to let the join operator to calculate a backw
Hi Rex,
There is a similar question asked recently which I think is the same reason
[1] called retraction amplification.
You can try to turn on the mini-batch optimization to reduce the retraction
amplification.
Best,
Jark
[1]:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Hi Dawid,
Just wanted to bump this thread in case you had any thoughts.
Thanks,
Steve
On Thu, Oct 29, 2020 at 2:42 PM Steve Whelan wrote:
> For some background, I am upgrading from Flink v1.9 to v1.11. So what I am
> about to describe is our implementation on v1.9, which worked. I am trying
>
Hey guys,
I am struggling to improve the throughput of my simple flink application.
The target topology is this.
read_from_kafka(byte array deserializer) --rescale-->
processFunction(confluent avro deserialization) -> split -> 1.
data_sink,2.dlq_sink
Kafka traffic is pretty high
Partitions: 128
T
Hi,
I am reading about the watermark creation of the kafka streams using the
article here:
https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/kafka.html#kafka-consumers-and-timestamp-extractionwatermark-emission
In there, it is a given example where the watermark assigner is di
Hi Till,
That's great! thank you so much!!! I have spent one week on this. I'm so
relieved!
Cheers
s
From: Till Rohrmann
Sent: 06 November 2020 17:56
To: Simone Cavallarin
Cc: user@flink.apache.org ; Aljoscha Krettek
Subject: Re: How to use properly the fu
