I can’t believe I’m suggesting this, but perhaps the Elasticsearch “Hammer of Thor” (aka “jar hell”) approach would be appropriate here.
Basically they prevent a program from running if there are duplicate classes on the classpath. This causes headaches when you really need a different version of library X, and that’s already on the class path. See https://github.com/elastic/elasticsearch/issues/14348 <https://github.com/elastic/elasticsearch/issues/14348> for an example of the issues it can cause. But it definitely catches a lot of oops-ish mistakes in building the jars, and makes debugging easier (they print out “class X jar1: <path to jar> jar2: <path to jar>”). > Caused by: java.lang.IllegalStateException: jar hell! > class: jdk.packager.services.UserJvmOptionsService > jar1: > /Library/Java/JavaVirtualMachines/jdk1.8.0_66.jdk/Contents/Home/lib/ant-javafx.jar > jar2: > /Library/Java/JavaVirtualMachines/jdk1.8.0_66.jdk/Contents/Home/lib/packager.jar — Ken > On Mar 9, 2018, at 3:21 AM, Stephan Ewen <se...@apache.org> wrote: > > Hi all! > > Flink 1.4 introduces child-first classloading by default, for the application > libraries. > > We added that, because it allows applications to use different versions of > many libraries, compared to what Flink uses in its core, or compared to what > other dependencies (like Hadoop) pull into the class path. > > For example, applications can use different versions of akka, Avro, Protobuf, > etc. Compared to what Flink / Hadoop / etc. uses. > > Now, while that is nice, child-first classloading runs into trouble when the > application jars are not properly built, meaning when the application JAR > contains libraries that it should not (because they are already in the > classpath / lib folder). > > For example, when the class path has the Kafka Connector (connector is in the > lib directory) and the application jar also contains Kafka, the we get nasty > errors due to class duplication and impossible class casts (X cannot be cast > to X). > > > What I would like to understand is how this change worked out for the users. > Based on that, we can keep this or revert this change in the next release. > > Please answer to this mail with: > > a. This was a great change, keep it and polish it. > > b. This caused in the end more problems than it solved, so please set the > default back to "parent-first" in 1.5 and leave "child-first" as an optional > flag. > > > Thanks a lot, > Stephan > -------------------------------------------- http://about.me/kkrugler +1 530-210-6378
