Hi all,
It's really a bug, haishui has created the FLINK-32496[1],
let's follow this issue there.
[1] https://issues.apache.org/jira/browse/FLINK-32496
Best,
Rui Fan
On Thu, Jun 29, 2023 at 8:34 PM Alexis Sarda-Espinosa <
sarda.espin...@gmail.com> wrote:
> BTW, it seems I spoke too soon in my
BTW, it seems I spoke too soon in my previous email. I left the job running
overnight with each source having its own alignment group to evaluate only
per-split alignment, and I can see that eventually some partitions never
resumed consumption and the consumer lag increased.
Regards,
Alexis.
Am D
Hi Martjin,
thanks for the pointers. I think the issue I'm seeing is not caused by
those because in my case the watermarks are not negative. Some more
information from my setup in case it's relevant:
- All Kafka topics have 6 partitions.
- Job parallelism is 2, but 2 of the Kafka sources are hard
Hi Alexis,
There are a couple of recent Flink tickets on watermark alignment,
specifically https://issues.apache.org/jira/browse/FLINK-32414 and
https://issues.apache.org/jira/browse/FLINK-32420 - Could the later be also
applicable in your case?
Best regards,
Martijn
On Wed, Jun 28, 2023 at 11:
Hello,
just for completeness, I don't see the problem if I assign a different
alignment group to each source, i.e. using only split-level watermark
alignment.
Regards,
Alexis.
Am Mi., 28. Juni 2023 um 08:13 Uhr schrieb haishui :
> Hi,
> I have the same trouble. This is really a bug.
> `shouldWa
Hello,
I am currently evaluating idleness and alignment with Flink 1.17.1 and the
externalized Kafka connector. My job has 3 sources whose watermark
strategies are defined like this:
WatermarkStrategy.forBoundedOutOfOrderness(maxAllowedWatermarkDrift)
.withIdleness(idleTimeout)
