Hello,
We’re trying to use a Parquet file sink to output files in s3.
When running in Streaming mode, it seems that parquet files are flushed and
rolled at each checkpoint. The result is a crazy high number of very small
parquet files which completely defeats the purpose of that format.
Is ther
Hey there,
We have some instabilities around checkpointing, that we don't quite
understand.
In general, as soon as a checkpoint fails, our cluster does not recover
back to a proper state.
But to better understand the mechanism, we'd like to be notified as soon as
this happens, so we can jump on ou
egards
> Lasse Nedergaard
>
>
> > Den 16. apr. 2021 kl. 18.29 skrev Mathieu D :
> >
> >
> > Hello,
> >
> > I'm totally new to Flink, and I'd like to make sure I understand things
> properly around watermarks.
> >
> > We're
Hello,
I'm totally new to Flink, and I'd like to make sure I understand things
properly around watermarks.
We're processing messages from iot devices.
Those messages have a timestamp, and we have a first phase of processing
based on this timestamp. So far so good.
These messages actually "pack"
