The conclusion in FLIP-196
<https://cwiki.apache.org/confluence/display/FLINK/FLIP-196%3A+Source+API+stability+guarantees>
was that we only provide source compatibility for Public(Evolving) APIs,
which means that _everything_ must be compiled against the Flink version
you are using it with.
Doesn't that mean that such dependency conflicts are then user errors?
On 10/02/2022 20:19, Thomas Weise wrote:
Hi Alexander,
It is beneficial for users to be able to replace/choose a connector as
part of their application. When flink-connector-base is included in
dist, that becomes more difficult. We can run into hard to understand
dependency conflicts such as [1]. Depending on the Flink platform
used, it may also be hard to update something that is part of dist. I
would prefer to keep the module outside dist.
Thanks,
Thomas
[1]https://lists.apache.org/thread/0ht5y2tyzpt16ft36zm428182dxfs3zx
On Wed, Feb 9, 2022 at 3:26 AM Alexander Fedulov
<alexan...@ververica.com> wrote:
Hi everyone,
I would like to discuss the best approach to address the issue raised
in FLINK-25927 [1]. It can be summarized as follows:
flink-connector-base is currently inconsistently used in connectors
(directly shaded in some and transitively pulled in via
flink-connector-files which is itself shaded in the table uber jar)
FLINK-24687 [2] moved flink-connector-files out of the flink-table uber
jar
It is necessary to make usage of flink-connector-base consistent across
all connectors
One approach is to stop shading flink-connector-files in all connectors and
instead package it in flink-dist, making it a part of Flink-wide provided
public API. This approach is investigated in the following PoC PR: 18545
[3]. The issue with this approach is that it breaks any existing CI and
IDE setups that do not directly rely on flink-dist and also do not include
flink-connector-files as an explicit dependency.
In theory, a nice alternative would be to make it a part of a dependency
that is ubiquitously provided, for instance, flink-streaming-java. Doing
that for flink-streaming-java would, however, introduce a dependency cycle
and is currently not feasible.
It would be great to hear your opinions on what could be the best way
forward here.
[1]https://issues.apache.org/jira/browse/FLINK-25927
[2]https://issues.apache.org/jira/browse/FLINK-24687
[3]https://github.com/apache/flink/pull/18545
Thanks,
Alexander Fedulov
