Maybe try including the jar with
--driver-class-path
> On Feb 26, 2015, at 12:16 PM, Akshat Aranya wrote:
>
> My guess would be that you are packaging too many things in your job, which
> is causing problems with the classpath. When your jar goes in first, you get
> the correct version o
My guess would be that you are packaging too many things in your job, which
is causing problems with the classpath. When your jar goes in first, you
get the correct version of protobuf, but some other version of something
else. When your jar goes in later, other things work, but protobuf
breaks.
Thanks for your response and suggestion, Sean. Setting
"spark.files.userClassPathFirst" didn't fix the problem for me. I am not
very familiar with the Spark and Scala environment, so please correct any
incorrect assumptions or statements I make.
However, I don't believe this to be a classpath visi
I assume this is a difference between your local driver classpath and
remote worker classpath. It may not be a question of whether the class
is there, but classpath visibility issues. Have you looked into
settings like spark.files.userClassPathFirst?
On Tue, Feb 24, 2015 at 4:43 AM, necro351 . wr
Sorry Ted, that was me clumsily trying to redact my organization's name
from the computer output (in my e-mail editor). I can assure you that
basically defend7 and rick are the same thing in this case so the class is
present in the jar.
On Mon Feb 23 2015 at 9:39:09 PM Ted Yu wrote:
> The classn
The classname given in stack trace was com.rick.reports.Reports
In the output from jar command the class is com.defend7.reports.Reports.
FYI
On Mon, Feb 23, 2015 at 9:33 PM, necro351 . wrote:
> Hi Ted,
>
> Yes it appears to be:
> rick@ubuntu:~/go/src/rick/sparksprint/containers/tests/Streaming
Hi Ted,
Yes it appears to be:
rick@ubuntu:~/go/src/rick/sparksprint/containers/tests/StreamingReports$
jar tvf
../../../analyzer/spark/target/scala-2.10/rick-processors-assembly-1.0.jar|grep
SensorReports
1128 Mon Feb 23 17:34:46 PST 2015
com/defend7/reports/Reports$SensorReports$1.class
13507
bq. Caused by: java.lang.ClassNotFoundException: com.rick.reports.Reports$
SensorReports
Is Reports$SensorReports class in rick-processors-assembly-1.0.jar ?
Thanks
On Mon, Feb 23, 2015 at 8:43 PM, necro351 . wrote:
> Hello,
>
> I am trying to deserialize some data encoded using proto buff fro
