Hi Lorenzo,
The main failure scenario that recommendation is addressing is when the
Flink job fails right after a checkpoint successfully completes, but before
the KafkaSink subtasks receive from the JM the checkpoint completed RPC
notification to commit the transactions. It is possible that durin
Hi team
In Kafka Sink docs [1], with EXACTLY_ONCE it is recommended to set:
transaction_timeout > maximum_checkpoint duration + maximum_restart_duration.
I understand transaction_timeout > maximum_checkpoint_duration
But why adding maximum_restart_duration?
If the application recovers from a ch
As per this link, it says that it only supports value_only for now as I am
using pyflink. Does it mean I can't extract the timestamp appended by Kafka
with pyflink as of now?
https://nightlies.apache.org/flink/flink-docs-release-1.17/docs/connectors/datastream/kafka/#deserializer
or does it mean s
Hi Team,
you can ignore this thread. I was able to resolve this.
J.
On Mon, Oct 2, 2023 at 8:40 PM joshua perez wrote:
> Hi team,
>
> I am trying to read the records from the Kafka topic and below is my very
> basic code as of now
>
> from pyflink.datastream.connectors.kafka import FlinkKafkaC
Hi team,
I am trying to read the records from the Kafka topic and below is my very
basic code as of now
from pyflink.datastream.connectors.kafka import FlinkKafkaConsumer
from pyflink.datastream.stream_execution_environment import
StreamExecutionEnvironment, RuntimeExecutionMode
from pyflink.comm
Hi everyone,
I’m using the code below to calculate the moving average on some data.
Table averageJudgeByPhaseReport = filteredPhasesDurationsTable
.window(Slide.over(lit(WINDOW_SIZE_IN_DAYS_REPORT).days())
.every(lit(WINDOW_SLIDE_IN_DAYS_REPORT).days())
.on($(“en
Hello folks,
Any help is appreciated.
J.
On Sat, Sep 30, 2023 at 1:47 PM joshua perez wrote:
> Hi Team,
>
> We recently have started a use case where there would be involvement of
> Kafka and Flink's low level APIs like map and process functions and since I
> am entirely new to these stuffs, I
Hi Liu and Jinfeng,
I am trying to implement KafkaDeserializationSchema for Pyflink but am
unable to get any examples. Can you share some links or references using
which I can understand and try to implement myself?
Perez sid.
