Can issues with Kafka consumption influence the watermark?

. There are 144 partitions on the input, parallelism 72, we use Kafka msg event timestamps (as set by the producer). We've seen it before with 72 partitions and parallelism of 72. -- Piotr Domagalski

Re-interepreting Kafka source as a keyed stream?

didn't notice any issues. -- Piotr Domagalski

Re: Watermark idleness and alignment - are they exclusive?

That looks exactly like what we hit, thank you! On Thu, Jun 15, 2023 at 10:57 PM Ken Krugler wrote: > I think you’re hitting this issue: > > https://issues.apache.org/jira/browse/FLINK-31632 > > Fixed in 1.16.2, 1.171. > > — Ken > > > On Jun 15, 2023, at 1:39

Watermark idleness and alignment - are they exclusive?

ore information on this? -- Piotr Domagalski

Re: Source vs SourceFunction and testing

urce(SourceFunction) and env.fromSource(Source) return a > DataStream, and you could apply downstream transformations onto it. > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface > > Cheers, > > Qingsheng > > > On May 25, 2

Re: Source vs SourceFunction and testing

ic is required to convert the consumer > records to what the workflow is expecting. > > — Ken > > On May 24, 2022, at 8:34 AM, Piotr Domagalski > wrote: > > Hi, > > I'm wondering: what ithe recommended way to structure the job which one > would like to te

Source vs SourceFunction and testing

k-training/blob/05791e55ad7ff0358b5c57ea8f40eada4a1f626a/ride-cleansing/src/test/java/org/apache/flink/training/exercises/ridecleansing/RideCleansingIntegrationTest.java#L61 -- Piotr Domagalski

At-least once sinks and their behaviour in a non-failure scenario

1.15/docs/deployment/cli/#stopping-a-job-gracefully-creating-a-final-savepoint -- Piotr Domagalski