Re: Proper way of adding external jars

2016-11-15 Thread Till Rohrmann
Which version of Flink are you using because I tested exactly this scenario with the latest 1.2-SNAPSHOT on a local flink cluster and it worked both ways (independent of which jar was specified by -C or provided as the user code jar). Can you maybe share the stack trace of the error. Then I could

Re: Proper way of adding external jars

2016-11-15 Thread Gyula Fóra
Hi Till, Sorry, I understand that this was confusing. The JarWithMain contains a class called Launcher with main method that does something like: Class.getForName("classFromUserJar").newInstance().launch() So it doesnt have any "static" dependency to the UserJar. JarWithMain has a lot of API cla

Re: Proper way of adding external jars

2016-11-15 Thread Till Rohrmann
Hi Gyula, did I understand it correct that JarWithMain depends on UserJar because the former will execute classes from the latter and UserJar depends on JarWithMain because it contains classes depending on class from JarWithMain? This sounds like a cyclic dependency to me. The command line help f

Re: Proper way of adding external jars

2016-11-15 Thread Gyula Fóra
Hi Scott, Thanks, I am familiar with the ways you suggested. Unfortunately packaging everything together is not really an option in our case, we specifically want to avoid having to do this as many people will set up their own builds and they will inevitable fail to include everything necessary wi

Re: Proper way of adding external jars

2016-11-14 Thread Scott Kidder
Hi Gyula, I've typically added external library dependencies to my own application JAR as shaded-dependencies. This ensures that all dependencies are included with my application while being distributed to Flink Job Manager & Task Manager instances. Another approach is to place these external JAR

Proper way of adding external jars

2016-11-14 Thread Gyula Fóra
Hi, I have been trying to use the -C flag to add external jars with user code and I have observed some strange behaviour. What I am trying to do is the following: I have 2 jars, JarWithMain.jar contains the main class and UserJar.jar contains some classes that the main method will eventually exec