[
https://issues.apache.org/jira/browse/SPARK-20818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Parente closed SPARK-20818.
---------------------------------
Resolution: Not A Problem
> Tab-to-autocomplete in IPython + Python3 results in job execution
> -----------------------------------------------------------------
>
> Key: SPARK-20818
> URL: https://issues.apache.org/jira/browse/SPARK-20818
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 2.0.2
> Reporter: Peter Parente
> Labels: ipython, jupyter
>
> Using Spark in a Jupyter Notebook 5.0 with an IPython 6.0 kernel and Python
> 3, when I press Tab to autocomplete the function names on a DataFrame or RDD,
> Spark executes the job graph constructed thus far. This only appears to
> happen for certain autocompletions, namely completions that resolve to
> functions having the @ignore_unicode_prefix decorator applied to them, and
> only in Python 3 (never in Python 2).
> https://github.com/apache/spark/blob/master/python/pyspark/rdd.py#L146
> Indeed, this function does have a special case for Python 3 in which it
> rewrites function docstrings. Why IPython autocompletion has a bad
> interaction with this logic is unknown (to me at least) at the moment.
> I reproduced this bug on Spark 2.0.2. The code in the decorator hasn't
> changed in 2.1.x, so the bug likely impacts that version as well.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
