add a critical bug https://issues.apache.org/jira/browse/SPARK-10474
(Aggregation failed with unable to acquire memory)
--
View this message in context:
http://apache-spark-developers-list.1001551.n3.nabble.com/VOTE-Release-Apache-Spark-1-5-0-RC3-tp13928p13987.html
Sent from the Apache Spark De
I saw a new "spark.shuffle.manager=tungsten-sort" implemented in
https://issues.apache.org/jira/browse/SPARK-7081, but it can't be found its
corresponding description in
http://people.apache.org/~pwendell/spark-releases/spark-1.5.0-rc3-docs/configuration.html(Currenlty
there are only 'sort' and 'ha
[mailto:r...@databricks.com]
Sent: Saturday, September 5, 2015 3:30 PM
To: Krishna Sankar
Cc: Davies Liu; Yin Huai; Tom Graves; dev@spark.apache.org
Subject: Re: [VOTE] Release Apache Spark 1.5.0 (RC3)
Thanks, Krishna, for the report. We should fix your problem using the Python
UDFs in 1.6 too.
I&#
Thanks, Krishna, for the report. We should fix your problem using the
Python UDFs in 1.6 too.
I'm going to close this vote now. Thanks everybody for voting. This vote
passes with 8 +1 votes (3 binding) and no 0 or -1 votes.
+1:
Reynold Xin*
Tom Graves*
Burak Yavuz
Michael Armbrust*
Davies Liu
For
Excellent & Thanks Davies. Yep, now runs fine and takes 1/2 the time !
This was exactly why I had put in the elapsed time calculations.
And thanks for the new pyspark.sql.functions.
+1 from my side for 1.5.0 RC3.
Cheers
On Fri, Sep 4, 2015 at 9:57 PM, Davies Liu wrote:
> Could you update the n
Could you update the notebook to use builtin SQL function month and year,
instead of Python UDF? (they are introduced in 1.5).
Once remove those two udfs, it runs successfully, also much faster.
On Fri, Sep 4, 2015 at 2:22 PM, Krishna Sankar wrote:
> Yin,
>It is the
> https://github.com/xsan
Yin,
It is the
https://github.com/xsankar/global-bd-conf/blob/master/004-Orders.ipynb.
Cheers
On Fri, Sep 4, 2015 at 9:58 AM, Yin Huai wrote:
> Hi Krishna,
>
> Can you share your code to reproduce the memory allocation issue?
>
> Thanks,
>
> Yin
>
> On Fri, Sep 4, 2015 at 8:00 AM, Krishna Sa
Krishna - I think the rename happened before rc1 actually. Was done couple
months ago.
On Fri, Sep 4, 2015 at 5:00 AM, Krishna Sankar wrote:
> Thanks Tom. Interestingly it happened between RC2 and RC3.
> Now my vote is +1/2 unless the memory error is known and has a workaround.
>
> Cheers
>
>
Hi Krishna,
Can you share your code to reproduce the memory allocation issue?
Thanks,
Yin
On Fri, Sep 4, 2015 at 8:00 AM, Krishna Sankar wrote:
> Thanks Tom. Interestingly it happened between RC2 and RC3.
> Now my vote is +1/2 unless the memory error is known and has a workaround.
>
> Cheers
Thanks Tom. Interestingly it happened between RC2 and RC3.
Now my vote is +1/2 unless the memory error is known and has a workaround.
Cheers
On Fri, Sep 4, 2015 at 7:30 AM, Tom Graves wrote:
> The upper/lower case thing is known.
> https://issues.apache.org/jira/browse/SPARK-9550
> I assume
The upper/lower case thing is known.
https://issues.apache.org/jira/browse/SPARK-9550I assume it was decided to be
ok and its going to be in the release notes but Reynold or Josh can probably
speak to it more.
Tom
On Thursday, September 3, 2015 10:21 PM, Krishna Sankar
wrote:
+
+1
Distinct count test is blazing fast - awesome!,
On Thu, Sep 3, 2015 at 8:21 PM Krishna Sankar wrote:
> +?
>
> 1. Compiled OSX 10.10 (Yosemite) OK Total time: 26:09 min
> mvn clean package -Pyarn -Phadoop-2.6 -DskipTests
> 2. Tested pyspark, mllib
> 2.1. statistics (min,max,mean,Pearson,
+?
1. Compiled OSX 10.10 (Yosemite) OK Total time: 26:09 min
mvn clean package -Pyarn -Phadoop-2.6 -DskipTests
2. Tested pyspark, mllib
2.1. statistics (min,max,mean,Pearson,Spearman) OK
2.2. Linear/Ridge/Laso Regression OK
2.3. Decision Tree, Naive Bayes OK
2.4. KMeans OK
Center And S
+1. Compiled on Windows with YARN and Hive. Tested Tungsten aggregation and
observed similar (good) performance comparing to 1.4 with unsafe on. Ran a
few workloads and tested SparkSQL thrift server
--
View this message in context:
http://apache-spark-developers-list.1001551.n3.nabble.com/VOTE-
+1, built 1.5 from source and ran TPC-DS locally and clusters, ran
performance benchmark for aggregation and join with difference scales,
all worked well.
On Thu, Sep 3, 2015 at 10:05 AM, Michael Armbrust
wrote:
> +1 Ran TPC-DS and ported several jobs over to 1.5
>
> On Thu, Sep 3, 2015 at 9:57 A
+1 Ran TPC-DS and ported several jobs over to 1.5
On Thu, Sep 3, 2015 at 9:57 AM, Burak Yavuz wrote:
> +1. Tested complex R package support (Scala + R code), BLAS and DataFrame
> fixes good.
>
> Burak
>
> On Thu, Sep 3, 2015 at 8:56 AM, mkhaitman
> wrote:
>
>> Built and tested on CentOS 7, Hado
+1. Tested complex R package support (Scala + R code), BLAS and DataFrame
fixes good.
Burak
On Thu, Sep 3, 2015 at 8:56 AM, mkhaitman wrote:
> Built and tested on CentOS 7, Hadoop 2.7.1 (Built for 2.6 profile),
> Standalone without any problems. Re-tested dynamic allocation specifically.
>
> "L
Built and tested on CentOS 7, Hadoop 2.7.1 (Built for 2.6 profile),
Standalone without any problems. Re-tested dynamic allocation specifically.
"Lost executor" messages are still an annoyance since they're expected to
occur with dynamic allocation, and shouldn't WARN/ERROR as they do now,
however
+1. Tested on Yarn with Hadoop 2.6.
A few of the things tested: pyspark, hive integration, aux shuffle handler,
history server, basic submit cli behavior, distributed cache behavior, cluster
and client mode...
Tom
On Tuesday, September 1, 2015 3:42 PM, Reynold Xin
wrote:
Please v
- As usual the license and signatures are OK
- No blockers, check
- 9 "Critical" bugs for 1.5.0 are listed below just for everyone's
reference (48 total issues still targeted for 1.5.0)
- Under Java 7 + Ubuntu 15, I only had one consistent test failure,
but obviously it's not failing in Jenkins
- I
Please vote on releasing the following candidate as Apache Spark version
1.5.0. The vote is open until Friday, Sep 4, 2015 at 21:00 UTC and passes
if a majority of at least 3 +1 PMC votes are cast.
[ ] +1 Release this package as Apache Spark 1.5.0
[ ] -1 Do not release this package because ...
To
21 matches
Mail list logo
