Path Style Access for S3 compliant object stores

2019-08-13 Thread Achyuth Narayan Samudrala
Hi user group, I am trying to use Flink to write to an S3 object store. I am using the flink-s3-fs-hadoop as a filesystem implementation to interact with this store. How do I provide the s3 property to enable path style access instead of virtual host addressing? I tried looking around in the docu

Flink fs s3 shaded hadoop: KerberosAuthException when using StreamingFileSink to S3

2019-08-08 Thread Achyuth Narayan Samudrala
Hi, We are trying to use StreamingFileSink to write to a S3 bucket. Its a simple job which reads from Kafka and sinks to S3. The credentials for s3 are configured in the flink cluster. We are using flink 1.7.2 without pre bundled hadoop. As suggested in the documentation we have added the flink-s3