I have opened the following issues to track new efforts to bring additional Azure Support in the following areas to the connectors ecosystem.
My goal is to add the first two features [FLINK-18562] and [FLINK-18568] to the existing file system capabilities [1] and then have the other connectors FLINK-1856[3-7] exist as standalone plugins. As more users adopt the additional connectors, we could incrementally bring them into the core code base if necessary with sufficient support. I am new to the process so that I have a few questions: Do I need to create a FLIP [2] in order to make these changes to bring the new capabilities or the individual JIRA issues are sufficient? I am thinking about targeting Flink versions 1.10 through 1.12 For new connectors like this, how many versions can/should this be integrated into? Are there any upcoming changes to supported Java and Scala versions that I need to be aware of? Any ideas or suggestions you have would be great. Below is a summary of the JIRA issues that were created to track the effort Add Support for Azure Data Lake Store Gen 2 in Flink File System https://issues.apache.org/jira/browse/FLINK-18562 Add Support for Azure Data Lake Store Gen 2 in Streaming File Sink https://issues.apache.org/jira/browse/FLINK-18568 Add Support for Azure Cosmos DB DataStream Connector https://issues.apache.org/jira/browse/FLINK-18563 Add Support for Azure Event Hub DataStream Connector https://issues.apache.org/jira/browse/FLINK-18564 Add Support for Azure Event Grid DataStream Connector https://issues.apache.org/jira/browse/FLINK-18565 Add Support for Azure Cognitive Search DataStream Connector https://issues.apache.org/jira/browse/FLINK-18566 Add Support for Azure Cognitive Search Table & SQL Connector https://issues.apache.org/jira/browse/FLINK-18567 [1] https://github.com/apache/flink/tree/master/flink-filesystems [2] https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals
