Re: Guaranteed processing orders of each batch in Spark Streaming

Hi, all: I have given a detailed description of my proposal in this jira . On Mon, Oct 19, 2015 at 2:58 PM Renjie Liu wrote: > Hi, all: > I've read source code and it seems that there is no guarantee that the > order of processing of each RDD is

Re: Ability to offer initial coefficients in ml.LogisticRegression

Hi Holden, Yep the issue id is correct. It seems that you're waiting for SPARK-11136 which Jayant is working on? Best, Yizhi 2015-11-03 11:14 GMT+08:00 Holden Karau : > Hi YiZhi, > > I've been waiting on the shared param to go in (I think it was kmeans) so we > could have a common API. I think t

Ability to offer initial coefficients in ml.LogisticRegression

Hi YiZhi, I've been waiting on the shared param to go in (I think it was kmeans) so we could have a common API. I think the issue is SPARK-7852 but I am on mobile right now. Cheers, Holden :) On Monday, November 2, 2015, DB Tsai > wrote: > Hi YiZhi, > > Sure. I think Holden already created a J

Re: Ability to offer initial coefficients in ml.LogisticRegression

Hi YiZhi, Sure. I think Holden already created a JIRA for this. Please coordinate with Holden, and keep me in the loop. Thanks. Sincerely, DB Tsai -- Web: https://www.dbtsai.com PGP Key ID: 0xAF08DF8D On Mon, Nov 2, 2015 at 7:32 AM, YiZhi

Re: test failed due to OOME

I have a PR which tries to address this issue: https://github.com/apache/spark/pull/9384 Comment is welcome. On Mon, Nov 2, 2015 at 9:53 AM, Patrick Wendell wrote: > I believe this is some bug in our tests. For some reason we are using way > more memory than necessary. We'll probably need to lo

[BUILD SYSTEM] quick jenkins downtime, november 5th 7am

i'd like to take jenkins down briefly thursday morning to install some plugin updates. this will hopefully be short (~1hr), but could easily become longer as the jenkins plugin ecosystem is fragile and updates like this are known to cause things to explode. the only reason why i'm contemplating t

Re: test failed due to OOME

I believe this is some bug in our tests. For some reason we are using way more memory than necessary. We'll probably need to log into Jenkins and heap dump some running tests and figure out what is going on. On Mon, Nov 2, 2015 at 7:42 AM, Ted Yu wrote: > Looks like SparkListenerSuite doesn't OO

Re: test failed due to OOME

Looks like SparkListenerSuite doesn't OOM on QA runs compared to Jenkins builds. I wonder if this is due to difference between machines running QA tests vs machines running Jenkins builds. On Fri, Oct 30, 2015 at 1:19 PM, Ted Yu wrote: > I noticed that the SparkContext created in each sub-test

Re: Ability to offer initial coefficients in ml.LogisticRegression

Hi Tsai, Is it proper if I create a jira and try to work on it? 2015-10-23 10:40 GMT+08:00 YiZhi Liu : > Thank you Tsai. > > Holden, would you mind posting the JIRA issue id here? I searched but > found nothing. Thanks. > > 2015-10-23 1:36 GMT+08:00 DB Tsai : >> There is a JIRA for this. I know H

Re: Downloading Hadoop from s3://spark-related-packages/

I am getting the same results using closer.lua versus close.cgi, which seems to be downloading a page where the user can choose the closest mirror. I tried to add parameters to follow redirect without much success. There seems to be already a jira for a similar request with infra: https://issues.ap

Re: Lead operator not working as aggregation operator

I get the part about using it with window, but most other window operators also work as aggregator operator and in this case, it is specifically mentioned in the jira issue as well. I asked on dev list and not user list as it was already mentioned in the issue. On Mon, Nov 2, 2015 at 4:15 PM, Herm

Re: Lead operator not working as aggregation operator

Hi, This is more a question for the User list. Lead and Lag imply ordering of the whole dataset, and this is not supported. You can use Lead/Lag in an ordered window function and you'll be fine: *select lead(max(expenses)) over (order by customerId) from tbl group by customerId* HTH Met vriend

Re: Lead operator not working as aggregation operator

I was referring to this jira issue : https://issues.apache.org/jira/browse/TAJO-919 On Mon, Nov 2, 2015 at 4:03 PM, Shagun Sodhani wrote: > Hi! I was trying out window functions in SparkSql (using hive context) > and I noticed that while this >

Lead operator not working as aggregation operator

Hi! I was trying out window functions in SparkSql (using hive context) and I noticed that while this mentions that *lead* is implemented as an aggregate operator, it seems not to be the case. I am using the followin

Re: Unable to run applications on spark in standalone cluster mode

The contents of spark-env.sh is : SPARK_MASTER_IP=marvin.spark.ins-01 SPARK_MASTER_PORT=7077 SPARK_MASTER_WEBUI_PORT=8080 SPARK_WORKER_WEBUI_PORT=8081 SPARK_WORKER_INSTANCES=1 SPARK_LOCAL_IP=marvin.spark.ins-01 The contents of etc/hosts is 172.28.161.33 marvin.base.ins-01

Re: Getting Started

wait, this is an identical email like was from "Aadi Thakar < thakkar.aa...@gmail.com>" a day before could it be a spambot? *Romi Kuntsman*, *Big Data Engineer* http://www.totango.com On Mon, Nov 2, 2015 at 10:12 AM, Romi Kuntsman wrote: > https://cwiki.apache.org/confluence/display/SPARK/Contr

Re: Getting Started

https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark *Romi Kuntsman*, *Big Data Engineer* http://www.totango.com On Fri, Oct 30, 2015 at 1:25 PM, Saurabh Shah wrote: > Hello, my name is Saurabh Shah and I am a second year undergraduate > student at DA-IICT, Gandhinagar, India