We have maven-enforcer-plugin defined in the pom. I don't know why it
didn't work for you. Could you try rebuild with maven2 and confirm
that there is no error message? If that is the case, please create a
JIRA for it. Thanks! -Xiangrui
On Wed, Jul 9, 2014 at 3:53 AM, Bharath Ravi Kumar wrote:
>
It seems to me a setup issue. I just tested news20.binary (1355191
features) on a 2-node EC2 cluster and it worked well. I added one line
to conf/spark-env.sh:
export SPARK_JAVA_OPTS=" -Dspark.akka.frameSize=20 "
and launched spark-shell with "--driver-memory 20g". Could you re-try
with an EC2 se
Hi Bharath,
1) Did you sync the spark jar and conf to the worker nodes after build?
2) Since the dataset is not large, could you try local mode first
using `spark-summit --driver-memory 12g --master local[*]`?
3) Try to use less number of partitions, say 5.
If the problem is still there, please a
The feature dimension is small. You don't need a big akka.frameSize.
The default one (10M) should be sufficient. Did you cache the data
before calling LRWithSGD? -Xiangrui
On Thu, Jul 3, 2014 at 10:02 AM, Bharath Ravi Kumar wrote:
> I tried another run after setting the driver memory to 8G (and
>
Could you check the driver memory in the executor tab of the Spark UI
when the job is running? If it is too small, please set
--driver-memory with spark-submit, e.g. 10g. Could you also attach the
master log under spark/logs as well? -Xiangrui
On Wed, Jul 2, 2014 at 9:34 AM, Bharath Ravi Kumar wr
Hi Xiangrui,
The issue with aggergating/counting over large feature vectors (as part of
LogisticRegressionWithSGD) continues to exist, but now in another form:
while the execution doesn't freeze (due to SPARK-1112), it now fails at the
second or third gradient descent iteration consistently with a
Thanks. I'll await the fix to re-run my test.
On Thu, Jun 19, 2014 at 8:28 AM, Xiangrui Meng wrote:
> Hi Bharath,
>
> This is related to SPARK-1112, which we already found the root cause.
> I will let you know when this is fixed.
>
> Best,
> Xiangrui
>
> On Tue, Jun 17, 2014 at 7:37 PM, Bharath
Hi Bharath,
This is related to SPARK-1112, which we already found the root cause.
I will let you know when this is fixed.
Best,
Xiangrui
On Tue, Jun 17, 2014 at 7:37 PM, Bharath Ravi Kumar wrote:
> Couple more points:
> 1)The inexplicable stalling of execution with large feature sets appears
>
Couple more points:
1)The inexplicable stalling of execution with large feature sets appears
similar to that reported with the news-20 dataset:
http://mail-archives.apache.org/mod_mbox/spark-user/201406.mbox/%3c53a03542.1010...@gmail.com%3E
2) The NPE trying to call mapToPair convert an RDD into a
Hi Xiangrui ,
I'm using 1.0.0.
Thanks,
Bharath
On 18-Jun-2014 1:43 am, "Xiangrui Meng" wrote:
> Hi Bharath,
>
> Thanks for posting the details! Which Spark version are you using?
>
> Best,
> Xiangrui
>
> On Tue, Jun 17, 2014 at 6:48 AM, Bharath Ravi Kumar
> wrote:
> > Hi,
> >
> > (Apologies f
Hi Bharath,
Thanks for posting the details! Which Spark version are you using?
Best,
Xiangrui
On Tue, Jun 17, 2014 at 6:48 AM, Bharath Ravi Kumar wrote:
> Hi,
>
> (Apologies for the long mail, but it's necessary to provide sufficient
> details considering the number of issues faced.)
>
> I'm ru
Hi,
(Apologies for the long mail, but it's necessary to provide sufficient
details considering the number of issues faced.)
I'm running into issues testing LogisticRegressionWithSGD a two node
cluster (each node with 24 cores and 16G available to slaves out of 24G on
the system). Here's a descrip
12 matches
Mail list logo
