Re: Using Python IDE for Spark Application Development

2014-08-06 Thread Mohit Singh
Take a look at this gist https://gist.github.com/bigaidream/40fe0f8267a80e7c9cf8 That worked for me. On Wed, Aug 6, 2014 at 7:32 PM, Sathish Kumaran Vairavelu < vsathishkuma...@gmail.com> wrote: > Mohit, This doesn't seems to be working can you please provide more > details? when I use "from pys

Re: Using Python IDE for Spark Application Development

2014-08-06 Thread Sathish Kumaran Vairavelu
Mohit, This doesn't seems to be working can you please provide more details? when I use "from pyspark import SparkContext" it is disabled in pycharm. I use pycharm community edition. Where should I set the environment variables in same python script or different python script? Also, should I run a

Re: Using Python IDE for Spark Application Development

2014-08-06 Thread Mohit Singh
My naive set up.. Adding os.environ['SPARK_HOME'] = "/path/to/spark" sys.path.append("/path/to/spark/python") on top of my script. from pyspark import SparkContext from pyspark import SparkConf Execution works from within pycharm... Though my next step is to figure out autocompletion and I bet the