Re: Spark dev-setup

2016-08-25 Thread Nishadi Kirielle
Thanks a lot for the guidelines. I could successfully configure and debug On Wed, Aug 24, 2016 at 7:05 PM, Jacek Laskowski wrote: > On Wed, Aug 24, 2016 at 2:32 PM, Steve Loughran > wrote: > > > no reason; the key thing is : not in cluster mode, as there your work > happens elsewhere > > Righ

Re: Spark dev-setup

2016-08-24 Thread Jacek Laskowski
On Wed, Aug 24, 2016 at 2:32 PM, Steve Loughran wrote: > no reason; the key thing is : not in cluster mode, as there your work happens > elsewhere Right! Anything but cluster mode should make it easy (that leaves us with local). Jacek --

Re: Spark dev-setup

2016-08-24 Thread Steve Loughran
> On 24 Aug 2016, at 11:38, Jacek Laskowski wrote: > > On Wed, Aug 24, 2016 at 11:13 AM, Steve Loughran > wrote: > >> I'd recommend > > ...which I mostly agree to with some exceptions :) > >> -stark spark standalone from there > > Why spark standalone since the OP asked about "learning how

Re: Spark dev-setup

2016-08-24 Thread Jacek Laskowski
On Wed, Aug 24, 2016 at 11:13 AM, Steve Loughran wrote: > I'd recommend ...which I mostly agree to with some exceptions :) > -stark spark standalone from there Why spark standalone since the OP asked about "learning how query execution flow occurs in Spark SQL"? How about spark-shell in local

Re: Spark dev-setup

2016-08-24 Thread Steve Loughran
On 24 Aug 2016, at 07:10, Nishadi Kirielle mailto:ndime...@gmail.com>> wrote: Hi, I'm engaged in learning how query execution flow occurs in Spark SQL. In order to understand the query execution flow, I'm attempting to run an example in debug mode with intellij IDEA. It would be great if anyon

Re: Spark dev-setup

2016-08-23 Thread Nishadi Kirielle
Hi, I'm engaged in learning how query execution flow occurs in Spark SQL. In order to understand the query execution flow, I'm attempting to run an example in debug mode with intellij IDEA. It would be great if anyone can help me with debug configurations. Thanks & Regards Nishadi On Tue, Jun 21,

Re: Spark dev-setup

2016-06-21 Thread Akhil Das
You can read this documentation to get started with the setup https://cwiki.apache.org/confluence/display/SPARK/Useful+Developer+Tools#UsefulDeveloperTools-IntelliJ There was a pyspark setup discussion on SO over here http://stackoverflow.com/questions/33478218/write-and-run-pyspark-in-intellij-i

Re: Spark Dev

2014-12-19 Thread Sandy Ryza
Hi Harikrishna, A good place to start is taking a look at the wiki page on contributing: https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark -Sandy On Fri, Dec 19, 2014 at 2:43 PM, Harikrishna Kamepalli < harikrishna.kamepa...@gmail.com> wrote: > > i am interested to contribu