Chesnay Schepler created FLINK-13602:
----------------------------------------
Summary: Presto S3 filesystem effectively does not support
credential providers
Key: FLINK-13602
URL: https://issues.apache.org/jira/browse/FLINK-13602
Project: Flink
Issue Type: Bug
Components: Connectors / FileSystem
Affects Versions: 1.9.0
Reporter: Chesnay Schepler
To provide credentials to S3 users may configure a credentials provider. For
providers from amazon (which are relocated) we allow users to configure the
original class name, and relocate it manually in the S3 filesystem factories.
However, none of the amazon provided credential providers can be used with the
Presto filesystem, since it _additionally_ requires them to have a constructor
accepting a hadoop configuration.
(https://prestodb.github.io/docs/current/connector/hive.html#amazon-s3-configuration)
{{hadoop-aws}} _does_ include a number of credential providers that have this
constructor, however these use configuration keys that aren't mirrored from the
flink config. (they expect {{fs.s3a}} as a key-prefix), not to mention that
users would have to configure the relocated class (since the S3 factory only
manually relocates amazon classes).
Finally, a custom implementation of the credentials provider can effectively be
ruled out since they too would have to be implemented against the relocated
amazon/hadoop classes, which we can't really expect users to do.
In summary, amazon providers aren't working since they don't have a constructor
that presto requires, hadoop providers don't work since we don't mirror the
required configuration keys, and custom providers are unreasonable as they'd
have to be implemented against relocated classes.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
