Yes, we've been recommending the use of the classloader option. I think it is a viable solution to the version clash problem. We're not very enthused about the classpath.first config because you could trade one set of problems for another.
Judging from some issues still being discovered (MAPREDUCE-5751 and MAPREDUCE-5813 for example), however, I feel like the classloader option is not being used or tested widely... On Fri, Mar 28, 2014 at 10:00 AM, Sandy Ryza <sandy.r...@cloudera.com>wrote: > My understanding is that unfortunately we're stuck with these for the rest > of 2.x, because changing them could break jobs that rely on them. For jobs > that want to use newer versions, the recommendation is to use > mapreduce.user.classpath.first or turn on classpath isolation with > mapreduce.job.classloader. > > -Sandy > > > On Fri, Mar 28, 2014 at 7:59 AM, Sangjin Lee <sjl...@gmail.com> wrote: > > > Hi folks, > > > > Even as 2.3 was released, several dependencies of hadoop are quite dated. > > And more and more of them are causing friction for hadoop-related > libraries > > and hadoop users in general, as these dependency versions are often > > incompatibly different than the versions most people use these days. So > > this is becoming a very real problem, if not one already. > > > > Some key ones include > > - guava: 11.0.2 (current 16.0.1) > > - jetty: 6.1.26 (current 9.1.3) > > - commons-lang: 2.6 (current 3.3.1) > > > > In particular, guava is already causing a lot of issues as many > developers > > have moved onto a newer version and are using APIs that do not exist in > > 11.0.2, and get NoSuchMethodErrors and such. > > > > Also, for jetty, 6.1.26 has been EOFed for some time now. > > > > Do we have a plan to review some of these dependencies and upgrade them > to > > reasonably up-to-date versions? I remember seeing some JIRAs on specific > > dependencies, but has a review been done to determine a good set of > > versions to upgrade to? > > > > It would be great if we could upgrade some of the more common ones at > least > > to modern versions. > > > > Thanks, > > Sangjin > > >
