Re: Passing arbitrary Hadoop s3a properties from FileSystem SQL Connector options

2021-12-15 Thread Arvid Heise
Hi Timothy, The issue would require a refactor FileSystems abstraction to allow multiple FileSystems objects of the same FileSystem type being configured differently. While this would improve the code quality and enable such use cases, I currently have no capacity to work on it or guide it. If yo

Re: Passing arbitrary Hadoop s3a properties from FileSystem SQL Connector options

2021-12-13 Thread Timothy James
Thank you Timo. Hi Arvid! I note that that ticket proposes two alternatives for solution. The first > Either we allow a properties map similar to Kafka or Kinesis properties to our connectors. seems to solve our problem. The second, much more detailed, appears unrelated to our needs: > Or so

Re: Passing arbitrary Hadoop s3a properties from FileSystem SQL Connector options

2021-12-13 Thread Timo Walther
Hi Timothy, unfortunetaly, this is not supported yet. However, the effort will be tracked under the following ticket: https://issues.apache.org/jira/browse/FLINK-19589 I will loop-in Arvid (in CC) which might help you in contributing the missing functioniality. Regards, Timo On 10.12.21