Live Code Review Experiment - Friday May 9th @ 11am pacific

[Holden Karau]

Hi Everyone,

In the interest of helping folks understand more about how our code reviews
are done (and maybe even encouraging more folks to contribute by helping
with code reviews), I'm going to try and do a live-streamed code review Friday
May 9th @ 11am pacific time

- if you're interested you will be able to see the code review stream here
. If the time doesn't work out
for you, the recording will be posted as well.

The comments will still end up the PR as always, so there is no need for
you to tune in unless the code review process is something your interested
in better understanding.

I already have one person who's opted into having their code reviewed live,
but if you have a PR in Python or ML and you wouldn't mind being a part of
this experiment please let me know.

Cheers,

Holden :)

-- 
Twitter: https://twitter.com/holdenkarau

Re: Live Code Review Experiment - Friday March 9th @ 11am pacific

[Holden Karau]

Oops, as pointed out to me I meant to say March 9th.

On Thu, Mar 1, 2018 at 2:19 PM, Holden Karau  wrote:

> Hi Everyone,
>
> In the interest of helping folks understand more about how our code
> reviews are done (and maybe even encouraging more folks to contribute by
> helping with code reviews), I'm going to try and do a live-streamed code
> review Friday March 9th @ 11am pacific time
> 
> - if you're interested you will be able to see the code review stream here
> . If the time doesn't work
> out for you, the recording will be posted as well.
>
> The comments will still end up the PR as always, so there is no need for
> you to tune in unless the code review process is something your interested
> in better understanding.
>
> I already have one person who's opted into having their code reviewed
> live, but if you have a PR in Python or ML and you wouldn't mind being a
> part of this experiment please let me know.
>
> Cheers,
>
> Holden :)
>
> --
> Twitter: https://twitter.com/holdenkarau
>



-- 
Twitter: https://twitter.com/holdenkarau

Using bundler for Jekyll?

[Holden Karau]

One of the things which comes up when folks update the Spark website is
that we often get lots of unnecessarily changed files. I _think_ some of
this might come from different jekyll versions on different machines, would
folks be OK if we added a requirements that folks use bundler so we can
have more consistent versions?

-- 
Twitter: https://twitter.com/holdenkarau

Re: Using bundler for Jekyll?

[Felix Cheung]

Also part of the problem is that the latest news panel is static on each page, 
so any new link added changes hundreds of files?


From: holden.ka...@gmail.com  on behalf of Holden Karau 

Sent: Thursday, March 1, 2018 6:36:43 PM
To: dev
Subject: Using bundler for Jekyll?

One of the things which comes up when folks update the Spark website is that we 
often get lots of unnecessarily changed files. I _think_ some of this might 
come from different jekyll versions on different machines, would folks be OK if 
we added a requirements that folks use bundler so we can have more consistent 
versions?

--
Twitter: https://twitter.com/holdenkarau

Re: Using bundler for Jekyll?

[Holden Karau]

Ah yeah bundler wouldn't fix that issue :( Do we happen to have any Jekyll
enthusiast lurking on the list who could help out?

On Thu, Mar 1, 2018 at 6:46 PM, Felix Cheung 
wrote:

> Also part of the problem is that the latest news panel is static on each
> page, so any new link added changes hundreds of files?
>
> --
> *From:* holden.ka...@gmail.com  on behalf of
> Holden Karau 
> *Sent:* Thursday, March 1, 2018 6:36:43 PM
> *To:* dev
> *Subject:* Using bundler for Jekyll?
>
> One of the things which comes up when folks update the Spark website is
> that we often get lots of unnecessarily changed files. I _think_ some of
> this might come from different jekyll versions on different machines, would
> folks be OK if we added a requirements that folks use bundler so we can
> have more consistent versions?
>
> --
> Twitter: https://twitter.com/holdenkarau
>



-- 
Twitter: https://twitter.com/holdenkarau

Re: Please keep s3://spark-related-packages/ alive

[Nicholas Chammas]

Marton,

Thanks for the tip. (Too bad the docs
 referenced from the issue
I opened with INFRA  make
no mention of mirrors.cgi.)

Matei,

A Requester Pays bucket is a good idea. I was trying to avoid having to
maintain a repository of assets, but I suppose it's ultimately unavoidable
given that Apache does not partner with a CDN. I will look into this for
Flintrock.

Nick

On Wed, Feb 28, 2018 at 7:21 AM Marton, Elek  wrote:

>
> >  2. *Apache mirrors are inconvenient to use.* When you download
> > something from an Apache mirror, you get a link like this one
> > <
> https://www.apache.org/dyn/closer.lua/spark/spark-2.2.1/spark-2.2.1-bin-hadoop2.7.tgz
> >.
> > Instead of automatically redirecting you to your download, though,
> > you need to process the results you get back
> > <
> https://github.com/nchammas/flintrock/blob/67bf84a1b7cfa1c276cf57ecd8a0b27613ad2698/flintrock/scripts/download-hadoop.py#L21-L42
> >
> > to find your download target. And you need to handle the high
> > download failure rate, since sometimes the mirror you get doesn’t
> > have the file it claims to have.
>
> It's not a full answer, just a note:
>
> You can also use mirrors.cgi instead of parsing the json from closer.lua:
>
>
> https://www.apache.org/dyn/mirrors/mirrors.cgi?action=download&filename=spark/spark-2.2.1/spark-2.2.1-bin-hadoop2.7.tgz
>
> (Unfortunatelly it doesn't check the availibility of the file. If it's
> moved to the archive you will be redirected to a 404)
>
> Marton
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>