Hi User group,
We are using spark Linear Regression with SGD as the optimization technique and
we are achieving very sub-optimal results.
Can anyone shed some light on why this implementation seems to produce such
poor results vs our own implementation?
We are using a very small dataset, but w
A colleague and I were having a discussion and we were disagreeing about
something in Spark/Mesos that perhaps someone can shed some light into.
We have a mesos cluster that runs spark via a sparkHome, rather than
downloading an executable and such.
My colleague says that say we have parquet fi
Yea, I wouldn't try and modify the current since RDDs are suppose to be
immutable, just create a new one...
val newRdd = oldRdd.map(r => (r._2(), r._1()))
or something of that nature...
Steve
From: Evo Eftimov [evo.efti...@isecc.com]
Sent: Thursday, May 14, 2015
2015 at 10:50 AM, Stephen Carman
mailto:scar...@coldlight.com>> wrote:
Sander,
I eventually solved this problem via the --[no-]switch_user flag, which is set
to true by default. I set this to false, which would have the user that owns
the process run the job, otherwise it was my username (sc
bably?).
>
> How did you install Mesos? Did you build from source yourself?
>
> Please install Mesos through a package or actually from source run make
> install and run from the installed binary.
>
> Tim
>
> On Mon, Apr 27, 2015 at 11:11 AM, Stephen Carman
> wrote:
>
So I installed spark on each of the slaves 1.3.1 built with hadoop2.6 I just
basically got the pre-built from the spark website…
I placed those compiled spark installs on each slave at /opt/spark
My spark properties seem to be getting picked up on my side fine…
[cid:683C1BA0-C9EC-448C-B1DB-E93A
So I can’t for the life of me to get something even simple working for Spark on
Mesos.
I installed a 3 master, 3 slave mesos cluster, which is all configured, but I
can’t for the life of me even get the spark shell to work properly.
I get errors like this
org.apache.spark.SparkException: Job ab
they can be made public inside the library or have some
interface to them such that children classes can make use of them?
Thanks,
Stephen Carman, M.S.
AI Engineer, Coldlight Solutions, LLC
Cell - 267 240 0363
This e-mail is intended solely for the above-mentioned recipient and it may
contain
