Re: cutting 1.6.2 rc and 2.0.0 rc this week?

Should we backport https://github.com/apache/spark/pull/13424 to 1.6.2 ? On Thu, Jun 16, 2016 at 9:02 AM andy petrella wrote: > +1 both too > (for tomorrow lunchtime? ^^) > > On Thu, Jun 16, 2016 at 5:06 AM Raymond Honderdors < > raymond.honderd...@sizmek.com> wrote: > >> +1 for both >> >> Get O

Re: cutting 1.6.2 rc and 2.0.0 rc this week?

+1 both too (for tomorrow lunchtime? ^^) On Thu, Jun 16, 2016 at 5:06 AM Raymond Honderdors < raymond.honderd...@sizmek.com> wrote: > +1 for both > > Get Outlook for Android > > > > On Wed, Jun 15, 2016 at 10:23 PM +0300, "Michael Armbrust" < > mich...@databricks.com> wrot

Re: cutting 1.6.2 rc and 2.0.0 rc this week?

+1 for both Get Outlook for Android On Wed, Jun 15, 2016 at 10:23 PM +0300, "Michael Armbrust" mailto:mich...@databricks.com>> wrote: +1 to both of these! On Wed, Jun 15, 2016 at 12:21 PM, Sean Owen mailto:so...@cloudera.com>> wrote: 1.6.2 RC seems fine to me; I don't

Yarn Log aggregation for a killed spark streaming job

Hi! We are seeing an issue around log aggregation under Yarn with Spark streaming. The specific case below is a example - we had to kill a spark streaming job, and would like to see the logs of the consumer so as to find out what happened before we had to kill it. Yarn reports the status of a kill

Re: Utilizing YARN AM RPC port field

FYI, I just filed https://issues.apache.org/jira/browse/SPARK-15974. Mingyu From: Mingyu Kim Date: Tuesday, June 14, 2016 at 2:13 PM To: Steve Loughran Cc: "dev@spark.apache.org" , Matt Cheah Subject: Re: Utilizing YARN AM RPC port field Thanks for the pointers, Steve! The first

Re: Hive-on-Spark with access to Hive from Spark jobs

Are you running Spark on YARN, Mesos, Standalone? For all of them you can make the Hive dependency just part of your application, and then you can manage this pretty easily. On Wed, Jun 15, 2016 at 2:35 AM, Rostyslav Sotnychenko < r.sotnyche...@gmail.com> wrote: > Hello! > > I have a question re

Re: cutting 1.6.2 rc and 2.0.0 rc this week?

+1 to both of these! On Wed, Jun 15, 2016 at 12:21 PM, Sean Owen wrote: > 1.6.2 RC seems fine to me; I don't know of outstanding issues. Clearly > we need to keep the 1.x line going for a bit, so a bug fix release > sounds good, > > Although we've got some work to do before 2.0.0 it does look li

Re: cutting 1.6.2 rc and 2.0.0 rc this week?

1.6.2 RC seems fine to me; I don't know of outstanding issues. Clearly we need to keep the 1.x line going for a bit, so a bug fix release sounds good, Although we've got some work to do before 2.0.0 it does look like it's within reach. Especially if declaring an RC creates more focus on resolving

Re: cutting 1.6.2 rc and 2.0.0 rc this week?

Any word on Kafka 0.10 support / SPARK-12177 I understand the hesitation, but is having nothing better than having a standalone subproject marked as experimental? On Wed, Jun 15, 2016 at 2:01 PM, Reynold Xin wrote: > It's been a while and we have accumulated quite a few bug fixes in > branch-1.6

cutting 1.6.2 rc and 2.0.0 rc this week?

It's been a while and we have accumulated quite a few bug fixes in branch-1.6. I'm thinking about cutting 1.6.2 rc this week. Any patches somebody want to get in last minute? On a related note, I'm thinking about cutting 2.0.0 rc this week too. I looked at the 60 unresolved tickets and almost all

ERROR RetryingBlockFetcher: Exception while beginning fetch of 1 outstanding blocks

I have a very simple driver which loads a textFile and filters a sub-string from each line in the textfile. When the collect action is executed , I am getting an exception. (The file is only 90 MB - so I am confused what is going on..) I am running on a local standalone cluster 16/06/15 19:45:22

Hive-on-Spark with access to Hive from Spark jobs

Hello! I have a question regarding Hive and Spark. As far as I know, in order to use Hive-on-Spark one need to compile Spark without Hive profile, but that means that it won't be possible to access Hive from normal Spark jobs. How is community going to address this issue? Making two different sp

Re: [YARN] Small fix for yarn.Client to use buildPath (not Path.SEPARATOR)

> On 14 Jun 2016, at 13:58, Jacek Laskowski wrote: > > Hi Steve and Sean, > > Didn't expect such a warm welcome from Sean and you! Since I'm with > Spark on YARN these days, let me see what I can do to make it nicer. > Thanks! > > I'm going to change Spark to use buildPath first. And then prop