Hi Jincheng,

Thanks a lot for your feedback!
Yes, I agree with you. There are cases that multi jars need to be uploaded.
I will prepare another discussion later. Maybe with a simple design doc.

Best, Hequn

On Wed, Jan 8, 2020 at 3:06 PM jincheng sun <sunjincheng...@gmail.com>
wrote:

> Thanks for bring up this discussion Hequn!
>
> +1 for include `flink-ml-api` and `flink-ml-lib` in opt.
>
> BTW: I think would be great if bring up a discussion for upload multiple
> Jars at the same time. as PyFlink JOB also can have the benefit if we do
> that improvement.
>
> Best,
> Jincheng
>
>
> Hequn Cheng <chenghe...@gmail.com> 于2020年1月8日周三 上午11:50写道:
>
> > Hi everyone,
> >
> > FLIP-39[1] rebuilds Flink ML pipeline on top of TableAPI which moves
> Flink
> > ML a step further. Base on it, users can develop their ML jobs and more
> and
> > more machine learning platforms are providing ML services.
> >
> > However, the problem now is the jars of flink-ml-api and flink-ml-lib are
> > only exist on maven repo. Whenever users want to submit ML jobs, they can
> > only depend on the ml modules and package a fat jar. This would be
> > inconvenient especially for the machine learning platforms on which
> nearly
> > all jobs depend on Flink ML modules and have to package a fat jar.
> >
> > Given this, it would be better to include jars of flink-ml-api and
> > flink-ml-lib in the `opt` folder, so that users can directly use the jars
> > with the binary release. For example, users can move the jars into the
> > `lib` folder or use -j to upload the jars. (Currently, -j only support
> > upload one jar. Supporting multi jars for -j can be discussed in another
> > discussion.)
> >
> > Putting the jars in the `opt` folder instead of the `lib` folder is
> because
> > currently, the ml jars are still optional for the Flink project by
> default.
> >
> > What do you think? Welcome any feedback!
> >
> > Best,
> >
> > Hequn
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs
> >
>

Reply via email to