Hi Robert, thanks for your reply. GlobalConfiguration.loadConfiguration was useful to check if a flink-conf.yml file was on resources, for the integration tests that I'm doing. On the cluster I will use the default configurations.
On Fri, Feb 21, 2020 at 10:58 AM Robert Metzger <rmetz...@apache.org> wrote: > There are multiple ways of passing configuration parameters to your user > defined code in Flink > > a) use getRuntimeContext().getUserCodeClassLoader().getResource() to > load a config file from your user code jar or the classpath. > b) use getRuntimeContext().getExecutionConfig().getGlobalJobParameters() to > access a configuration object serialized from the main method. > you can pass a custom object to the job parameters, or use Flink's > "Configuration" object in your main method: > > final StreamExecutionEnvironment env = > StreamExecutionEnvironment.getExecutionEnvironment(); > > Configuration config = new Configuration(); > config.setString("foo", "bar"); > env.getConfig().setGlobalJobParameters(config); > > c) Load the flink-conf.yaml: > > Configuration conf = GlobalConfiguration.loadConfiguration(); > > I'm not 100% sure if this approach works, as it is not intended to be used > in user code (I believe). > > > Let me know if this helps! > > Best, > Robert > > On Thu, Feb 20, 2020 at 1:50 PM Chesnay Schepler <ches...@apache.org> > wrote: > >> First things first, we do not intend for users to use anything in the S3 >> filesystem modules except the filesystems itself, >> meaning that you're somewhat treading on unsupported ground here. >> >> Nevertheless, the S3 modules contain a large variety of AWS-provided >> CerentialsProvider implementations, >> that can derive credentials from environment variables, system >> properties, files on the classpath and many more. >> >> Ultimately though, you're kind of asking us how to use AWS APIs, for >> which I would direct you to the AWS documentation. >> >> On 20/02/2020 13:16, David Magalhães wrote: >> >> I'm using >> org.apache.flink.fs.s3base.shaded.com.amazonaws.client.builder.AwsClientBuilder >> to create a S3 client to copy objects and delete object inside >> a TwoPhaseCommitSinkFunction. >> >> Shouldn't be another way to set up configurations without put them >> hardcoded ? Something like core-site.xml or flink-conf.yaml ? >> >> Right now I need to have them hardcoded like this. >> >> AmazonS3ClientBuilder.standard >> .withPathStyleAccessEnabled(true) >> .withEndpointConfiguration( >> new EndpointConfiguration("http://minio:9000", "us-east-1") >> ) >> .withCredentials( >> new AWSStaticCredentialsProvider(new BasicAWSCredentials("minio", >> "minio123")) >> ) >> .build >> >> Thanks >> >> >>