I would be in favor of limiting the scope here. The problem you might run into is that FinalizableReferenceQueue <https://code.google.com/p/guava-libraries/source/browse/guava/src/com/google/common/base/FinalizableReferenceQueue.java?name=v10.0.1&r=45768c3232017d0f123cf17f03b36941adf71e19> uses the system classloader and I ran into weird issues if you don't share those classes across the boundary.
On Thu, Nov 5, 2015 at 3:51 PM, Marcelo Vanzin <[email protected]> wrote: > On Thu, Nov 5, 2015 at 3:41 PM, Joey Paskhay <[email protected]> > wrote: > > We verified the Guava libraries are in the huge list of the included > jars, > > but we saw that in the > > org.apache.spark.sql.hive.client.IsolatedClientLoader.isSharedClass > method > > it seems to assume that *all* "com.google" (excluding "com.google.cloud") > > classes should be loaded from the base class loader. The Spark libraries > > seem to have *some* "com.google.common.base" classes shaded in but not > all. > > Yeah, seems to me like HiveContext should not be trying to include > guava in the shared list at all; the goal is to not have any Guava > classes show up in Spark's classpath, unfortunately that's currently > not possible because some types are exposed in the Java API (the ones > that are not shaded). > > Could you file a bug to track this? > > > -- > Marcelo > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
