Re: [RESULT] [VOTE] Release Apache Spark 1.1.0 (RC4)

Hey just a heads up to everyone - running a bit behind on getting the final artifacts and notes up. Finalizing this release was much more complicated than previous ones due to new binary formats (we need to redesign the download page a bit for this to work) and the large increase in contributor cou

Re: yet another jenkins restart early thursday morning -- 730am PDT (and a brief update on our new jenkins infra)

that's kinda what we're hoping as well. :) On Wed, Sep 10, 2014 at 2:46 PM, Nicholas Chammas < nicholas.cham...@gmail.com> wrote: > I'm looking forward to this. :) > > Looks like Jenkins is having trouble triggering builds for new commits or > after user requests (e.g. >

Re: yet another jenkins restart early thursday morning -- 730am PDT (and a brief update on our new jenkins infra)

I'm looking forward to this. :) Looks like Jenkins is having trouble triggering builds for new commits or after user requests (e.g. ). Hopefully that will be resolved tomorrow. Nick On Tue, Sep 9, 2014 at 5:00 PM, shane knapp wrot

Re: Lost executor on YARN ALS iterations

That's right On Tue, Sep 9, 2014 at 2:04 PM, Debasish Das wrote: > Last time it did not show up on environment tab but I will give it another > shot...Expected behavior is that this env variable will show up right ? > > On Tue, Sep 9, 2014 at 12:15 PM, Sandy Ryza > wrote: > >> I would expect 2

Re: parquet predicate / projection pushdown into unionAll

Hey Cody, Thanks for doing this! Will look at your PR later today. Michael On Wed, Sep 10, 2014 at 9:31 AM, Cody Koeninger wrote: > Tested the patch against a cluster with some real data. Initial results > seem like going from one table to a union of 2 tables is now closer to a > doubling of

Re: parquet predicate / projection pushdown into unionAll

Tested the patch against a cluster with some real data. Initial results seem like going from one table to a union of 2 tables is now closer to a doubling of query time as expected, instead of 5 to 10x. Let me know if you see any issues with that PR. On Wed, Sep 10, 2014 at 8:19 AM, Cody Koeninge

Re: parquet predicate / projection pushdown into unionAll

So the obvious thing I was missing is that the analyzer has already resolved attributes by the time the optimizer runs, so the references in the filter / projection need to be fixed up to match the children. Created a PR, let me know if there's a better way to do it. I'll see about testing perfor