Hi,
the state is checkpointed in subdirectories and with unique file names, so
having all in one root directory is no problem. This all happens automatically.
As far as I know, there is no implementation that generates output paths for
sinks like that. You could open a jira with a feature wish,
Hi Stefan,
In the past, I ran four separate Flink apps to sink data from four separate
Kafka topics to s3 without any transformations applied. For each Flink app,
I would set the checkpoint directory
to s3://some-bucket/checkpoints/topic-name. It appears that with Flink 1.4
I can just use a regex
> On Jan 3, 2018, at 5:51 AM, Stefan Richter
> wrote:
>
> Hi,
>
> first, let my ask why you want to have a different checkpoint directory per
> topic? It is perfectly ok to have just a single checkpoint directory, so I
> wonder what the intention is? Flink will already create proper subdire
Hi,
first, let my ask why you want to have a different checkpoint directory per
topic? It is perfectly ok to have just a single checkpoint directory, so I
wonder what the intention is? Flink will already create proper subdirectories
and filenames and can identify the right checkpoint data for e
Flink 1.4 added regex pattern matching for FlinkKafkaConsumer's which is a
neat feature. I would like to use this feature, but I'm wondering how that
impacts the FsStateBackend checkpointing mechanism. Before I would
subscribe to one topic and set a checkpoint path specific to that topic for
exampl
