FYI, we have an existing ApplicationClassLoader implementation that is used to isolate client/task classes from the rest. If we're going down the route of classloader isolation on this, it would be good to come up with a coherent strategy regarding both of these.
As a more practical step, I like the idea of isolating usage of guava that breaks with guava 16 and later. I assume (but I haven't looked into it) that it's fairly straightforward to isolate them and fix them. That work could be done at any time without any version upgrades or impacting users. On Mon, Nov 10, 2014 at 9:26 AM, Alejandro Abdelnur <tuc...@gmail.com> wrote: > IMO we should: > > 1* have a clean and thin client API JAR (which does not drag any 3rd party > dependencies, or a well defined small set -i.e. slf4j & log4j-) > 2* have a client implementation that uses a classloader to isolate client > impl 3rd party deps from app dependencies. > > #2 can be done using a stock URLClassLoader (i would just subclass it to > forbid packages in the API JAR and exposed 3rd parties to be loaded from > the app JAR) > > #1 is the tricky thing as our current API modules don't have a clean > API/impl separation. > > thx > PS: If folks are interested in pursing this, I can put together a prototype > of how #2 would work (I don't think it will be more than 200 lines of > code) > > > On Mon, Nov 10, 2014 at 5:18 AM, Steve Loughran <ste...@hortonworks.com> > wrote: > > > Yes, Guava is a constant pain; there's lots of open JIRAs related to it, > as > > its the one we can't seamlessly upgrade. Not unless we do our own fork > and > > reinsert the missing classes. > > > > The most common uses in the code are > > > > @VisibleForTesting (easily replicated) > > and the Precondition.check() operations > > > > The latter is also easily swapped out, and we could even add the check > they > > forgot: > > Preconditions.checkArgNotNull(argname, arg) > > > > > > These are easy; its the more complex data structures that matter more. > > > > I think for Hadoop 2.7 & java 7 we need to look at this problem and do > > something. Even if we continue to ship Guava 11 so that the HBase team > > don't send any (more) death threats, we can/should rework Hadoop to build > > and run against Guava 16+ too. That's needed to fix some of the recent > java > > 7/8+ changes. > > > > -Everything in v11 dropped from v16 MUST to be implemented with our own > > versions. > > -anything tagged as deprecated in 11+ SHOULD be replaced by newer stuff, > > wherever possible. > > > > I think for 2.7+ we should add some new profiles to the POM, for Java 8 > and > > 9 alongside the new baseline java 7. For those later versions we could > > perhaps mandate Guava 16. > > > > > > > > On 10 November 2014 00:42, Arun C Murthy <a...@hortonworks.com> wrote: > > > > > ... has been a constant pain w.r.t compatibility etc. > > > > > > Should we consider adopting a policy to not use guava in > > Common/HDFS/YARN? > > > > > > MR doesn't matter too much since it's application-side issue, it does > > hurt > > > end-users though since they still might want a newer guava-version, but > > at > > > least they can modify MR. > > > > > > Thoughts? > > > > > > thanks, > > > Arun > > > > > > > > > -- > > > CONFIDENTIALITY NOTICE > > > NOTICE: This message is intended for the use of the individual or > entity > > to > > > which it is addressed and may contain information that is confidential, > > > privileged and exempt from disclosure under applicable law. If the > reader > > > of this message is not the intended recipient, you are hereby notified > > that > > > any printing, copying, dissemination, distribution, disclosure or > > > forwarding of this communication is strictly prohibited. If you have > > > received this communication in error, please contact the sender > > immediately > > > and delete it from your system. Thank You. > > > > > > > -- > > CONFIDENTIALITY NOTICE > > NOTICE: This message is intended for the use of the individual or entity > to > > which it is addressed and may contain information that is confidential, > > privileged and exempt from disclosure under applicable law. If the reader > > of this message is not the intended recipient, you are hereby notified > that > > any printing, copying, dissemination, distribution, disclosure or > > forwarding of this communication is strictly prohibited. If you have > > received this communication in error, please contact the sender > immediately > > and delete it from your system. Thank You. > > >
