Thanks for bring up this discussion Hequn! +1 for include `flink-ml-api` and `flink-ml-lib` in opt.
BTW: I think would be great if bring up a discussion for upload multiple Jars at the same time. as PyFlink JOB also can have the benefit if we do that improvement. Best, Jincheng Hequn Cheng <chenghe...@gmail.com> 于2020年1月8日周三 上午11:50写道： > Hi everyone, > > FLIP-39[1] rebuilds Flink ML pipeline on top of TableAPI which moves Flink > ML a step further. Base on it, users can develop their ML jobs and more and > more machine learning platforms are providing ML services. > > However, the problem now is the jars of flink-ml-api and flink-ml-lib are > only exist on maven repo. Whenever users want to submit ML jobs, they can > only depend on the ml modules and package a fat jar. This would be > inconvenient especially for the machine learning platforms on which nearly > all jobs depend on Flink ML modules and have to package a fat jar. > > Given this, it would be better to include jars of flink-ml-api and > flink-ml-lib in the `opt` folder, so that users can directly use the jars > with the binary release. For example, users can move the jars into the > `lib` folder or use -j to upload the jars. (Currently, -j only support > upload one jar. Supporting multi jars for -j can be discussed in another > discussion.) > > Putting the jars in the `opt` folder instead of the `lib` folder is because > currently, the ml jars are still optional for the Flink project by default. > > What do you think? Welcome any feedback! > > Best, > > Hequn > > [1] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs >
