Re: Using filesystem plugin with MiniCluster

2023-01-12 Thread Chesnay Schepler
There is no good way in 1.15 IIRC. Adding a dependency on flink-s3-fs-hadoop _can_ work, if you dont run into dependency conflicts. Otherwise you have to create a plugin manager yourself, point it to some local directory via a system property (I think?), and then eagerly call FileSystem#init

Re: Using filesystem plugin with MiniCluster

2023-01-03 Thread Yaroslav Tkachenko
Hey Ken, I have flink-s3-fs-hadoop as a provided dependency in my project, and I've configured my IDE to include provided dependencies when starting applications. Works just fine. On Tue, Jan 3, 2023 at 5:06 PM Ken Krugler wrote: > Hi all, > > With Flink 1.15.x, is there a way to use the S3 Pre

Using filesystem plugin with MiniCluster

2023-01-03 Thread Ken Krugler
Hi all, With Flink 1.15.x, is there a way to use the S3 Presto plugin when running code on the MiniCluster? I can’t just add that jar as a dependency when testing, as I get: java.lang.NoClassDefFoundError: Could not initialize class com.facebook.presto.hive.s3.PrestoS3FileSystem at or