BTW, just out of curiosity, I checked both the 1.3.0 release assembly and the spark-core_2.10 artifact downloaded from http://mvnrepository.com/, and neither contain any references to anything under "org.eclipse" (all referenced jetty classes are the shaded ones under org.spark-project.jetty).
On Mon, Apr 6, 2015 at 10:30 PM, Josh Rosen <rosenvi...@gmail.com> wrote: > My hunch is that this behavior was introduced by a patch to start shading > Jetty in Spark 1.3: https://issues.apache.org/jira/browse/SPARK-3996. > > Note that Spark's MetricsSystem class is marked as private[spark] and thus > isn't intended to be interacted with directly by users. It's not super > likely that this API would break, but it's excluded from our MiMa checks and > thus is liable to change in incompatible ways across releases. > > If you add these Jetty classes as a compile-only dependency but don't add > them to the runtime classpath, do you get runtime errors? If the metrics > system is usable at runtime and we only have errors when attempting to > compile user code against non-public APIs, then I'm not sure that this is a > high-priority issue to fix since. If the metrics system doesn't work at > runtime, on the other hand, then that's definitely a bug that should be > fixed. > > If you'd like to continue debugging this issue, I think we should move this > discussion over to JIRA so it's easier to track and reference. > > Hope this helps, > Josh > > On Thu, Apr 2, 2015 at 7:34 AM, Jacek Lewandowski > <jacek.lewandow...@datastax.com> wrote: >> >> A very simple example which works well with Spark 1.2, and fail to compile >> with Spark 1.3: >> >> build.sbt: >> >> name := "untitled" >> version := "1.0" >> scalaVersion := "2.10.4" >> libraryDependencies += "org.apache.spark" %% "spark-core" % "1.3.0" >> >> Test.scala: >> >> package org.apache.spark.metrics >> import org.apache.spark.SparkEnv >> class Test { >> SparkEnv.get.metricsSystem.report() >> } >> >> Produces: >> >> Error:scalac: bad symbolic reference. A signature in MetricsSystem.class >> refers to term eclipse >> in package org which is not available. >> It may be completely missing from the current classpath, or the version on >> the classpath might be incompatible with the version used when compiling >> MetricsSystem.class. >> >> Error:scalac: bad symbolic reference. A signature in MetricsSystem.class >> refers to term jetty >> in value org.eclipse which is not available. >> It may be completely missing from the current classpath, or the version on >> the classpath might be incompatible with the version used when compiling >> MetricsSystem.class. >> >> This looks like something wrong with shading jetty. >> MetricsSystem references MetricsServlet which references some classes from >> Jetty, in the original package instead of shaded one. I'm not sure, but >> adding the following dependencies solves the problem: >> >> libraryDependencies += "org.eclipse.jetty" % "jetty-server" % >> "8.1.14.v20131031" >> libraryDependencies += "org.eclipse.jetty" % "jetty-servlet" % >> "8.1.14.v20131031" >> >> Is it intended or is it a bug? >> >> >> Thanks ! >> >> >> Jacek >> >> > -- Marcelo --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
