Sorry to spam people who are not interested. Greatly appreciate it if anyone who is familiar with this can share some insights.
On Wed, Jul 6, 2016 at 2:28 PM Chen Song <chen.song...@gmail.com> wrote: > Hi > > I ran into problems to use class loader in Spark. In my code (run within > executor), I explicitly load classes using the ContextClassLoader as below. > > Thread.currentThread().getContextClassLoader() > > The jar containing the classes to be loaded is added via the --jars option > in spark-shell/spark-submit. > > I always get the class not found exception. However, it seems to work if I > compile these classes in main jar for the job (the jar containing the main > job class). > > I know Spark implements its own class loaders in a particular way. Is > there a way to work around this? In other words, what is the proper way to > programmatically load classes in other jars added via --jars in Spark? > >
