Re: [Pyspark 2.3+] Timeseries with Spark

2019-06-13 Thread Jörn Franke
Time series can mean a lot of different things and algorithms. Can you describe more what you mean by time series use case, ie what is the input, what do you like to do with the input and what is the output? > Am 14.06.2019 um 06:01 schrieb Rishi Shah : > > Hi All, > > I have a time series use

[Pyspark 2.3+] Timeseries with Spark

2019-06-13 Thread Rishi Shah
Hi All, I have a time series use case which I would like to implement in Spark... What would be the best way to do so? Any built in libraries? -- Regards, Rishi Shah

Spark on Yarn - Dynamically getting a list of archives from --archives in spark-submit

2019-06-13 Thread Tommy Li
Hi Is there any way to get a list of the archives submitted with a spark job from the spark context? I see that spark context has a `.files()` function which returns the files included with `--files`, but I don't see an equivalent for `--archives`. Thanks, Tommy

Re: [Spark Core]: What is the release date for Spark 3 ?

2019-06-13 Thread Vadim Semenov
next spark summit On Thu, Jun 13, 2019 at 3:58 AM Alex Dettinger wrote: > Follow up on the release date for Spark 3. Any guesstimate or rough > estimation without commitment would be helpful :) > > Cheers, > Alex > > On Mon, Jun 10, 2019 at 5:24 PM Alex Dettinger > wrote: > >> Hi guys, >> >>

Re: Exposing JIRA issue types at GitHub PRs

2019-06-13 Thread Dongjoon Hyun
Thank you for the feedbacks and requirements, Hyukjin, Reynold, Marco. Sure, we can do whatever we want. I'll wait for more feedbacks and proceed to the next steps. Bests, Dongjoon. On Wed, Jun 12, 2019 at 11:51 PM Marco Gaido wrote: > Hi Dongjoon, > Thanks for the proposal! I like the idea.

Re: best docker image to use

2019-06-13 Thread Marcelo Valle
Thanks Riccardo. This is useful, and it seems it's maintained by jupyter team. I was hoping I would find some maintained by spark team. Right now, I am using the base images from this repo: https://github.com/big-data-europe/docker-spark/ -Marcelo On Tue, 11 Jun 2019 at 12:19, Riccardo Ferrari

Re: [Spark Core]: What is the release date for Spark 3 ?

2019-06-13 Thread Alex Dettinger
Follow up on the release date for Spark 3. Any guesstimate or rough estimation without commitment would be helpful :) Cheers, Alex On Mon, Jun 10, 2019 at 5:24 PM Alex Dettinger wrote: > Hi guys, > > I was not able to find the foreseen release date for Spark 3. > Would one have any informat