Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2554#issuecomment-57508231
Thanks for the very thorough description of this issue. It looks like
`IPYTHON=1 ./bin/pyspark` works as expected in Spark 1.0.2 and 1.1.0, so it
looks like this is only an issue in the master branch.
I think that the original motivation for the `IPYTHON=1` flag was that
older versions of IPython didn't use `PYTHONSTARTUP`, so this required us to
use the `%run` magic to load the PySpark shell's startup file. I think
`IPYTHON=1` was added as a conveinence method and to avoid writing code to
detect whether a particular Python executable was IPython (in retrospect, we
probably should have just hidden this complexity from users and performed that
auto-detection). In January, it looks like we removed support for IPython <
1.0 (82a1d38aea3b10930a2659b9c0e7ad2fb2c2ab4a).
The approach in this PR is very nice, since we no longer require special
handling / detection of IPython. This looks good to me, too, so I'd like to
merge it (pending Jenkins).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
