FWIW, I tested the first rc and saw no regressions. I ran our benchmarks
built against spark 1.3 and saw results consistent with spark 1.2/1.2.1.
On 2/25/15, 5:51 PM, "Patrick Wendell" wrote:
>Hey All,
>
>Just a quick updated on this thread. Issues have continued to trickle
>in. Not all of them
Hey All,
Just a quick updated on this thread. Issues have continued to trickle
in. Not all of them are blocker level but enough to warrant another
RC:
I've been keeping the JIRA dashboard up and running with the latest
status (sorry, long link):
https://issues.apache.org/jira/issues/?jql=project%
Hey all,
I found a major issue where JobProgressListener (a listener used to keep
track of jobs for the web UI) never forgets stages in one of its data
structures. This is a blocker for long running applications.
https://issues.apache.org/jira/browse/SPARK-5967
I am testing a fix for this right n
+1 (non-binding)
For: https://issues.apache.org/jira/browse/SPARK-3660
. Docs OK
. Example code is good
-Soumitra.
On Mon, Feb 23, 2015 at 10:33 AM, Marcelo Vanzin
wrote:
> Hi Tom, are you using an sbt-built assembly by any chance? If so, take
> a look at SPARK-5808.
>
> I haven't had any pr
My bad, had once fixed all Hive 12 test failures in PR #4107, but didn't
got time to get it merged.
Considering the release is close, I can cherry-pick those Hive 12 fixes
from #4107 and open a more surgical PR soon.
Cheng
On 2/24/15 4:18 AM, Michael Armbrust wrote:
On Sun, Feb 22, 2015 at
Nothing that I can point to, so this may only be a problem in test scope.
I am looking at a problem where some UDFs that run with 0.12 fail with
0.13; but that problem is already present in Spark 1.2.x, so it's not a
blocking regression for 1.3. (Very likely a HiveFunctionWrapper serde
problem, bu
On Sun, Feb 22, 2015 at 11:20 PM, Mark Hamstra
wrote:
> So what are we expecting of Hive 0.12.0 builds with this RC? I know not
> every combination of Hadoop and Hive versions, etc., can be supported, but
> even an example build from the "Building Spark" page isn't looking too good
> to me.
>
I
Hi Tom, are you using an sbt-built assembly by any chance? If so, take
a look at SPARK-5808.
I haven't had any problems with the maven-built assembly. Setting
SPARK_HOME on the executors is a workaround if you want to use the sbt
assembly.
On Fri, Feb 20, 2015 at 2:56 PM, Tom Graves
wrote:
> Try
Hey Patrick,
Do you have a link to the bug related to Python and Yarn? I looked at
the blockers in Jira but couldn't find it.
On Mon, Feb 23, 2015 at 10:18 AM, Patrick Wendell wrote:
> So actually, the list of blockers on JIRA is a bit outdated. These
> days I won't cut RC1 unless there are no k
It's only been reported on this thread by Tom, so far.
On Mon, Feb 23, 2015 at 10:29 AM, Marcelo Vanzin wrote:
> Hey Patrick,
>
> Do you have a link to the bug related to Python and Yarn? I looked at
> the blockers in Jira but couldn't find it.
>
> On Mon, Feb 23, 2015 at 10:18 AM, Patrick Wendel
So actually, the list of blockers on JIRA is a bit outdated. These
days I won't cut RC1 unless there are no known issues that I'm aware
of that would actually block the release (that's what the snapshot
ones are for). I'm going to clean those up and push others to do so
also.
The main issues I'm a
Thanks Sean. I glossed over the comment about SPARK-5669.
On Mon, Feb 23, 2015 at 9:05 AM, Sean Owen wrote:
> Yes my understanding from Patrick's comment is that this RC will not
> be released, but, to keep testing. There's an implicit -1 out of the
> gates there, I believe, and so the vote won'
Yes my understanding from Patrick's comment is that this RC will not
be released, but, to keep testing. There's an implicit -1 out of the
gates there, I believe, and so the vote won't pass, so perhaps that's
why there weren't further binding votes. I'm sure that will be
formalized shortly.
FWIW he
This vote was supposed to close on Saturday but it looks like no PMCs voted
(other than the implicit vote from Patrick). Was there a discussion offline
to cut an RC2? Was the vote extended?
On Mon, Feb 23, 2015 at 6:59 AM, Robin East wrote:
> Running ec2 launch scripts gives me the following err
Running ec2 launch scripts gives me the following error:
ssl.SSLError: [Errno 1] _ssl.c:504: error:14090086:SSL
routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
Full stack trace at
https://gist.github.com/insidedctm/4d41600bc22560540a26
I’m running OSX Mavericks 10.9.5
I’ll inves
So what are we expecting of Hive 0.12.0 builds with this RC? I know not
every combination of Hadoop and Hive versions, etc., can be supported, but
even an example build from the "Building Spark" page isn't looking too good
to me.
Working from f97b0d4, the example build command works: mvn -Pyarn
-
Trying to run pyspark on yarn in client mode with basic wordcount example I see
the following error when doing the collect:
Error from python worker: /usr/bin/python: No module named sqlPYTHONPATH was:
/grid/3/tmp/yarn-local/usercache/tgraves/filecache/20/spark-assembly-1.3.0-hadoop2.6.0.1.1411
+1 (non-binding)
- Verified signatures using [1]
- Built on MacOSX Yosemite
- Built on Fedora 21
Each build was run with and Hadoop-2.4 version with yarn, hive, and
hive-thriftserver profiles
I am having trouble getting all the tests passing on a single run on both
machines but we have this same
+1 (non-binding)
Tested Mesos coarse/fine-grained mode with 4 nodes Mesos cluster with
simple shuffle/map task.
Will be testing with more complete suite (ie: spark-perf) once the
infrastructure is setup to do so.
Tim
On Thu, Feb 19, 2015 at 12:50 PM, Krishna Sankar wrote:
> Excellent. Explicit
Excellent. Explicit toDF() works.
a) employees.toDF().registerTempTable("Employees") - works
b) Also affects saveAsParquetFile - orders.toDF().saveAsParquetFile
Adding to my earlier tests:
4.0 SQL from Scala and Python
4.1 result = sqlContext.sql("SELECT * from Employees WHERE State = 'WA'") OK
4.
>
> P.S: For some reason replacing "import sqlContext.createSchemaRDD" with "
> import sqlContext.implicits._" doesn't do the implicit conversations.
> registerTempTable
> gives syntax error. I will dig deeper tomorrow. Has anyone seen this ?
We will write up a whole migration guide before the f
+1 (non-binding, of course)
1. Compiled OSX 10.10 (Yosemite) OK Total time: 14:50 min
mvn clean package -Pyarn -Dyarn.version=2.6.0 -Phadoop-2.4
-Dhadoop.version=2.6.0 -Phive -DskipTests -Dscala-2.11
2. Tested pyspark, mlib - running as well as compare results with 1.1.x &
1.2.x
2.1. statisti
On Wed, Feb 18, 2015 at 6:13 PM, Patrick Wendell wrote:
>> Patrick this link gives a 404:
>> https://people.apache.org/keys/committer/pwendell.asc
>
> Works for me. Maybe it's some ephemeral issue?
Yes works now; I swear it didn't before! that's all set now. The
signing key is in that file.
> UISeleniumSuite:
> *** RUN ABORTED ***
> java.lang.NoClassDefFoundError: org/w3c/dom/ElementTraversal
> ...
This is a newer test suite. There is something flaky about it, we
should definitely fix it, IMO it's not a blocker though.
>
> Patrick this link gives a 404:
> https://people.apache.org
On OS X and Ubuntu I see the following test failure in the source
release for 1.3.0-RC1:
UISeleniumSuite:
*** RUN ABORTED ***
java.lang.NoClassDefFoundError: org/w3c/dom/ElementTraversal
...
Patrick this link gives a 404:
https://people.apache.org/keys/committer/pwendell.asc
Finally, I alrea
25 matches
Mail list logo
