Hi,
adding question from user group to dev group need expert advice
please help us decide which version to choose for production as standard.
http://apache-spark-user-list.1001560.n3.nabble.com/Status-of-2-11-support-tp25362.html
thanks
--
Thanks & Regards
Sachin Aggarwal
7760502772
I'm afraid you're correct, Krishna:
core/src/main/scala/org/apache/spark/package.scala: val SPARK_VERSION =
"1.6.0-SNAPSHOT"
docs/_config.yml:SPARK_VERSION: 1.6.0-SNAPSHOT
On Mon, Dec 14, 2015 at 6:51 PM, Krishna Sankar wrote:
> Guys,
>The sc.version gives 1.6.0-SNAPSHOT. Need to change to
+spark user mailing list
Hi there,
I have exactly the same problem as mentioned below. My current work around
is to add the jar containing my UDP in one of the system classpath (for
example, put it under the same path as
/opt/cloudera/parcels/CDH-5.4.2-1.cdh5.4.2.p0.2/jars/parquet-hadoop-bundle-1
Spar SQL's in-memory cache stores statistics per column which in turn is
used to skip batches(default size 1) within partition
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/ColumnStats.scala#L25
Hope this helps
Thanks
-Nitin
On T
Guys,
The sc.version gives 1.6.0-SNAPSHOT. Need to change to 1.6.0. Can you pl
verify ?
Cheers
On Sat, Dec 12, 2015 at 9:39 AM, Michael Armbrust
wrote:
> Please vote on releasing the following candidate as Apache Spark version
> 1.6.0!
>
> The vote is open until Tuesday, December 15, 2015 at
that looks like the lintr checks failed, causing the build to fail.
On Mon, Dec 14, 2015 at 3:05 PM, Yin Huai wrote:
> Hi Shane,
>
> Seems Spark's lint-r started to fail from
> https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/Spark-Master-SBT/4260/AMPLAB_JENKINS_BUI
Hi Shane,
Seems Spark's lint-r started to fail from
https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/Spark-Master-SBT/4260/AMPLAB_JENKINS_BUILD_PROFILE=hadoop1.0,label=spark-test/console.
Is it related to the upgrade work of R?
Thanks,
Yin
On Mon, Dec 14, 2015 at
...and we're back. we were getting reverse proxy timeouts, which seem
to have been caused by jenkins churning and doing a lot of IO. i'll
dig in to the logs and see if i can find out what happened.
weird.
shane
On Mon, Dec 14, 2015 at 11:51 AM, shane knapp wrote:
> something is up w/apache.
something is up w/apache. looking.
On Mon, Dec 14, 2015 at 11:37 AM, shane knapp wrote:
> after killing and restarting jenkins, things seem to be VERY slow.
> i'm gonna kick jenkins again and see if that helps.
>
>
>
> On Mon, Dec 14, 2015 at 11:26 AM, shane knapp wrote:
>> ok, we're back up an
+1 (non-binding)
Tested some workloads using basic API and DataFrame API on my 4-nodes
YARN cluster (1 master and 3 slaves).
I also tested the Web UI.
(I'm resending this mail just in case because it seems that I failed to
send the mail to dev@)
On 2015/12/13 2:39, Michael Armbrust wrote:
Pl
after killing and restarting jenkins, things seem to be VERY slow.
i'm gonna kick jenkins again and see if that helps.
On Mon, Dec 14, 2015 at 11:26 AM, shane knapp wrote:
> ok, we're back up and building.
>
> On Mon, Dec 14, 2015 at 10:31 AM, shane knapp wrote:
>> last week i forgot to downgr
ok, we're back up and building.
On Mon, Dec 14, 2015 at 10:31 AM, shane knapp wrote:
> last week i forgot to downgrade R to 3.1.1, and since there's not much
> activity right now, i'm going to take jenkins down and finish up the
> ticket.
>
> https://issues.apache.org/jira/browse/SPARK-11255
>
>
+1
Ran PageRank on standalone mode with 4 nodes and noticed a speedup after
the specific commits that were in RC2 but not RC1:
c247b6a Dec 10 [SPARK-12155][SPARK-12253] Fix executor OOM in unified
memory management
05e441e Dec 9 [SPARK-12165][SPARK-12189] Fix bugs in eviction of storage
memory by
Hello!
I'm currently working on POC and try to use Random Forest (classification
and regression). I also have to check SVM and Multiclass perceptron (other
algos are less important at the moment). So far I've discovered that Random
Forest has a limitation of maxDepth for trees and just out of curi
+1
Tested some dataframe operations on my Mac.
On Saturday, December 12, 2015, Michael Armbrust
wrote:
> Please vote on releasing the following candidate as Apache Spark version
> 1.6.0!
>
> The vote is open until Tuesday, December 15, 2015 at 6:00 UTC and passes
> if a majority of at least 3 +
last week i forgot to downgrade R to 3.1.1, and since there's not much
activity right now, i'm going to take jenkins down and finish up the
ticket.
https://issues.apache.org/jira/browse/SPARK-11255
we should be back up and running within 30 minutes.
thanks!
shane
--
++joshrosen
This Is Known[tm], and we have a bug open against it:
https://issues.apache.org/jira/browse/SPARK-11823
On Mon, Dec 14, 2015 at 7:42 AM, Ted Yu wrote:
> Attached was the tail of test suite output from local run.
> I got test failure.
>
> FYI
>
> On Sun, Dec 13, 2015 at 10:03 PM, Yin
Here are a fixed version of the docs for 1.6:
http://people.apache.org/~pwendell/spark-releases/spark-1.6.0-rc2-docsfixed-docs
There still might be some minor rendering issues of the ML page, but people
are investigating.
On Sat, Dec 12, 2015 at 6:58 PM, Burak Yavuz wrote:
> +1 tested SparkSQL
Hi,
As a part of the project, we are trying to create parallel implementation
of BIRCH clustering algorithm [1]. We are mostly getting idea how to do it
from this paper, which used CUDA to make BIRCH parallel [2]. ([2] is short
paper, just section 4. is relevant).
We would like to implement BIRCH
Attached was the tail of test suite output from local run.
I got test failure.
FYI
On Sun, Dec 13, 2015 at 10:03 PM, Yin Huai wrote:
> Can you reproduce the problem in your local environment? Our 1.6 hadoop
> 2.4 maven build looks pretty good. Since our 1.6 is pretty close to master,
> I am won
I've read through the mail archives and read the different threads.
I believe there is a great deal of value in teaching others.
I'm a 14yr vet of Java and would like to contribute to different Spark
projects. Here are my dilemmas:
1)How does one quickly get a working environment up and
With Java 7 / Ubuntu 15, and "-Pyarn -Phadoop-2.6 -Phive
-Phive-thriftserver", I still see the Docker tests fail every time. Is
anyone else seeing them fail (or running them)?
The Hive CliSuite also fails (stack trace at the bottom).
Same deal -- if people are running this test and it's not faili
+1 (non binding)
Tested our workloads on a standalone cluster:
- Spark Core
- Spark SQL
- Spark MLlib
- Python API
On 12 December 2015 at 18:39, Michael Armbrust
wrote:
> Please vote on releasing the following candidate as Apache Spark version
> 1.6.0!
>
> The vote is open until Tuesday, Dece
Thanks Shivaram, created https://issues.apache.org/jira/browse/SPARK-12318
I will work on it.
On Mon, Dec 14, 2015 at 4:13 PM, Shivaram Venkataraman <
shiva...@eecs.berkeley.edu> wrote:
> I think its just a bug -- I think we originally followed the Python
> API (in the original PR [1]) but the Py
I think its just a bug -- I think we originally followed the Python
API (in the original PR [1]) but the Python API seems to have been
changed to match Scala / Java in
https://issues.apache.org/jira/browse/SPARK-6366
Feel free to open a JIRA / PR for this.
Thanks
Shivaram
[1] https://github.com/
25 matches
Mail list logo
