Maybe try including the jar with
--driver-class-path
> On Feb 26, 2015, at 12:16 PM, Akshat Aranya wrote:
>
> My guess would be that you are packaging too many things in your job, which
> is causing problems with the classpath. When your jar goes in first, you get
> the correct version o
My guess would be that you are packaging too many things in your job, which
is causing problems with the classpath. When your jar goes in first, you
get the correct version of protobuf, but some other version of something
else. When your jar goes in later, other things work, but protobuf
breaks.
Hello everyone,
We are trying to decode a message inside a Spark job that we receive from
Kafka. The message is encoded using Proto Buff. The problem is when
decoding we get class-not-found exceptions. We have tried remedies we found
online in Stack Exchange and mail list archives but nothing seem
Thanks for your response and suggestion, Sean. Setting
"spark.files.userClassPathFirst" didn't fix the problem for me. I am not
very familiar with the Spark and Scala environment, so please correct any
incorrect assumptions or statements I make.
However, I don't believe this to be a classpath visi
I assume this is a difference between your local driver classpath and
remote worker classpath. It may not be a question of whether the class
is there, but classpath visibility issues. Have you looked into
settings like spark.files.userClassPathFirst?
On Tue, Feb 24, 2015 at 4:43 AM, necro351 . wr
Sorry Ted, that was me clumsily trying to redact my organization's name
from the computer output (in my e-mail editor). I can assure you that
basically defend7 and rick are the same thing in this case so the class is
present in the jar.
On Mon Feb 23 2015 at 9:39:09 PM Ted Yu wrote:
> The classn
The classname given in stack trace was com.rick.reports.Reports
In the output from jar command the class is com.defend7.reports.Reports.
FYI
On Mon, Feb 23, 2015 at 9:33 PM, necro351 . wrote:
> Hi Ted,
>
> Yes it appears to be:
> rick@ubuntu:~/go/src/rick/sparksprint/containers/tests/Streaming
Hi Ted,
Yes it appears to be:
rick@ubuntu:~/go/src/rick/sparksprint/containers/tests/StreamingReports$
jar tvf
../../../analyzer/spark/target/scala-2.10/rick-processors-assembly-1.0.jar|grep
SensorReports
1128 Mon Feb 23 17:34:46 PST 2015
com/defend7/reports/Reports$SensorReports$1.class
13507
bq. Caused by: java.lang.ClassNotFoundException: com.rick.reports.Reports$
SensorReports
Is Reports$SensorReports class in rick-processors-assembly-1.0.jar ?
Thanks
On Mon, Feb 23, 2015 at 8:43 PM, necro351 . wrote:
> Hello,
>
> I am trying to deserialize some data encoded using proto buff fro
Hello,
I am trying to deserialize some data encoded using proto buff from within
Spark and am getting class-not-found exceptions. I have narrowed the
program down to something very simple that shows the problem exactly (see
'The Program' below) and hopefully someone can tell me the easy fix :)
So
10 matches
Mail list logo
