Re: [DISCUSS] Inverted (child-first) class loading

2018-03-12 Thread Ken Krugler
Hi Kedar, See some thoughts inline below. I will admit that classpath issues (as in “I can see my damn class in the file, I can load it from my code, but I’m getting a freakin’ class not found exception at runtime…arghhh”) have been one of the biggest hassles with Flink-based projects that we’

Re: [DISCUSS] Inverted (child-first) class loading

2018-03-12 Thread kedar mhaswade
Many thanks Aljoscha! I am sorry I missed this section. Regards, Kedar On Mon, Mar 12, 2018 at 9:16 AM, Aljoscha Krettek wrote: > Hi Kedar, > > There is this section in the Flink docs: https://ci.apache.org/ > projects/flink/flink-docs-master/monitoring/debugging_classloading.html > > Best, > A

Re: [DISCUSS] Inverted (child-first) class loading

2018-03-12 Thread Aljoscha Krettek
Hi Kedar, There is this section in the Flink docs: https://ci.apache.org/projects/flink/flink-docs-master/monitoring/debugging_classloading.html Best, Aljoscha > On 10. Mar 2018, at 05:53, kedar mh

Re: [DISCUSS] Inverted (child-first) class loading

2018-03-10 Thread kedar mhaswade
This is an interesting question and it usually has consequences that are far-reaching in user experience. If a Flink app is supposed to be a "standalone app" that any Flink installation should be able to run, then the child-first classloading makes sense. This is how we build many of the Java appl

Re: [DISCUSS] Inverted (child-first) class loading

2018-03-09 Thread Stephan Ewen
@Ken very interesting thought. One for have three options: - forbid duplicate classes - parent first conflict resolution - child first conflict resolution Having number one as the default and let the error message suggest options two and three as options would definitely make users aware of

Re: [DISCUSS] Inverted (child-first) class loading

2018-03-09 Thread Ken Krugler
I can’t believe I’m suggesting this, but perhaps the Elasticsearch “Hammer of Thor” (aka “jar hell”) approach would be appropriate here. Basically they prevent a program from running if there are duplicate classes on the classpath. This causes headaches when you really need a different version

Re:[DISCUSS] Inverted (child-first) class loading

2018-03-09 Thread mingleizhang
Hi, Stephan It is a great change, keep it and polish it. nice nice nice I think the more fewer NoClassDefFoundError or ClassNotFoundException the user will encounter in the future. But I would like to two question about this functionality. If Im am wrong, please helps me out. Thank you in adv

[DISCUSS] Inverted (child-first) class loading

2018-03-09 Thread Stephan Ewen
Hi all! Flink 1.4 introduces child-first classloading by default, for the application libraries. We added that, because it allows applications to use different versions of many libraries, compared to what Flink uses in its core, or compared to what other dependencies (like Hadoop) pull into the c