Hi,
We are trying to upgrade to Flink 1.7.1. We have a job with parallelism > 1
over 2 tasks. Due to implementation details outside our control one of
these tasks cannot share the same JVM with another parallel instance i.e.
if the parallelism is 4 we need 4 slots across different Task Managers.
On 20 July 2017 at 17:54, Aljoscha Krettek wrote:
> You said you cancel and restart the job. How do you then restart the Job?
> From a savepoint or externalised checkpoint? Do you also see missing data
> when using an externalised checkpoint or a savepoint?
>
> Best,
> Aljoscha
>
Forgot to add that when a job gets cancelled via the UI (this is not the
case when the Yarn session is killed) a part file ending in ".pending" does
appear in S3, but that never seems to be promoted to finished upon restart
of the job
On 20 July 2017 at 11:41, Francisco Blaya
wr
of the offsets itself and
> includes these in the checkpoints. In case of a recovery, it does not rely
> on the offsets which were committed back to Kafka but only on the offsets
> it checkpointed itself.
> Gordon (in CC) is familiar with all details of Flink's Kafka consumer and
Hi,
We have a Flink job running on AWS EMR sourcing a Kafka topic and
persisting the events to S3 through a DateTimeBucketer. We configured the
bucketer to flush to S3 with an inactivity period of 5 mins.The rate at
which events are written to Kafka in the first place is very low so it is
easy for
