Yes, I enabled checkpointing and now the files do not have .pending extension.
Thank you Urs.
On Saturday, September 2, 2017, 3:10:28 AM PDT, Urs Schoenenberger
wrote:
Urs Schoenenberger (urs.schoenenber...@tngtech.com) is not on your Guest List
| Approve sender | Approve domain
Hi,
Hi,
you need to enable checkpointing for your job. Flink uses ".pending"
extensions to mark parts that have been completely written, but are not
included in a checkpoint yet.
Once you enable checkpointing, the .pending extensions will be removed
whenever a checkpoint completes.
Regards,
Urs
On
BTW, I am using a BucketingSink and a DateTimeBucketer. Do I need to set any
other property to move the files from .pending state.
BucketingSink sink = new
BucketingSink("hdfs://localhost:8020/flinktwitter/");sink.setBucketer(new
DateTimeBucketer("-MM-dd--HHmm"));
On Friday, September
