Given that the Table API is going to be Flink's main API for analytical workloads, it makes sense to make it as easy as possible for users to actually use it.
The question I would have is which other transitive dependencies are we gonna add to Flink's system class path by adding the Table API jars to /lib. I would assume that most if not all should be filtered out by the child-first class loader. However, if some unshaded dependencies come with the Table API jars and users have disabled the child first class loading, then we might break setups with this change. A release note could mitigate this problem. Cheers, Till On Tue, Jun 11, 2019 at 5:29 PM Stephan Ewen <se...@apache.org> wrote: > Hi all! > > I want to discuss making the Table API jars part of the "flink uber jar" in > "/lib" by default. > > So far, the Table API was an optional dependency in "/opt". > With the current effort to make it a first class API in Flink, it would be > good experience if the Table API was available by default. > > There are a few open questions, though: > > (1) Java only, or Scala as well? > ==> Ideally the Scala Table API is no runtime dependency, but only a > client-side (pre-flight) dependency and simply part of the user program, > not part of Flink's "/lib" > > (2) Which runner? Flink or Blink? > ==> Ideally both > ==> Do we need to rename the Blink classes to have ".blink." in the > package to avoid class name clashes? > ==> Do we need to shade/relocate much to avoid dependency clashes? > > (3) What should happen with the legacy BatchTableEnvironment (DataSet > based) module? > ==> Should be possible to simply add this as well, if the Flink runner > is included anyways. > > Note that this does not mean users add a dependency to everything when they > develop, so they should not see all environments. > It simply should make table programs executable without moving stuff from > /opt to /lib > > Best, > Stephan >
