the sort is logged my first instinct would be that the operation is being slowed down, _a lot_. Where are you archiving them to? Could it be the write operation is being throttled heavily? On 21/11/2019 13:48, Pavel Potseluev wrote:Hi Vino, Usually Flink archives jobs correctly and the problem i
sing idle slot [498b9bf0c0f2188ff739d72e6df288dc]. 21.11.2019, 06:07, "vino yang" :If everything is OK(your config options about archive dir and history server is correct), Flink should archive the completed job. You said you did not find any exceptions in the log about failing to archiv
Hi all, We see occasionally that flink doesn't save information about canceled job to archive directory (configured by jobmanager.archive.fs.dir property). And there are no exceptions in the log about failing archiving. It's a problem in our use case because our script for deploying jobs relies on
ou're using RocksDB with full checkpoint and the incremental checkpoint size is as small as not exceeding your expected throttle for S3, you could directly try incremental checkpoint to resolve the current problem. Thanks. Best Regards,Yu On Fri, 12 Jul 2019 at 20:39, Pavel Potseluev <pots
Hello! We use flink with periodically checkpointing to s3 file system. And when flink uploads checkpoint to s3 it makes high load to the network. We have found in AWS CLI S3 configuration option called max_bandwidth which allows to limit rate in bytes per second. Is there a way to have the same fun
Hi! We use flink-1.7.1 and have some problems with restoring from savepoint. We use custom kryo serializer which relies on protobuf representation of our model classes. It had been working fine but when we made some change in our model class it broke because of changed serialVersionUID. We can see
