Re: Flink filesystem S3, not finding the plugin

2021-10-30 Thread Arvid Heise
Hi Gil, you should not need to call FileSystem.initialize. The only entry point where it's currently necessary is the LocalExecutionEnvironment [1] but that's a bug. As done now, you are actually circumventing the plugin manager, so I'm not suprised that it's not working. [1] https://issues.apach

Flink filesystem S3, not finding the plugin

2021-10-29 Thread Gil De Grove
Hello, We are currently developing a RichParallelSourceFunction<> that reads from different FileSystem dynamically based on the configuration provided when starting the job. When running the tests, adding the hadoop-s3-presto library in the classpath, we can run the workload without any issues. H