I don't think it makes sense to deprecate or drop support for Python 2.7 until at least 2020, when 2.7 itself will be EOLed. (As of Spark 2.0, Python 2.6 support is deprecated and will be removed by Spark 2.2. Python 2.7 is only version of Python 2 that's still fully supported.)
Given the widespread industry use of Python 2.7, and the fact that it is supported upstream by the Python core developers until 2020, I don't see why Spark should even consider dropping support for it before then. There is, of course, additional ongoing work to support Python 2.7, but it seems more than justified by its level of use and popularity in the broader community. And I say that as someone who almost exclusively develops in Python 3.5+ these days. Perhaps by 2018 the industry usage of Python 2 will drop precipitously and merit a discussion about dropping support, but I think at this point it's premature to discuss that and we should just wait and see. Nick On Sun, Dec 4, 2016 at 10:59 AM Maciej Szymkiewicz <mszymkiew...@gmail.com> wrote: > Hi, > > I am aware there was a previous discussion about dropping support for > different platforms ( > http://apache-spark-developers-list.1001551.n3.nabble.com/Straw-poll-dropping-support-for-things-like-Scala-2-10-td19553.html) > but somehow it has been dominated by Scala and JVM and never touched the > subject of Python 2. > > Some facts: > > - Python 2 End Of Life is scheduled for 2020 ( > http://legacy.python.org/dev/peps/pep-0373/) without with "no > guarantee that bugfix releases will be made on a regular basis" until then. > - Almost all commonly used libraries already support Python 3 ( > https://python3wos.appspot.com/). A single exception that can be > important for Spark is thrift (Python 3 support is already present on the > master) and transitively PyHive and Blaze. > - Supporting both Python 2 and Python 3 introduces significant > technical debt. In practice Python 3 is a different language with backward > incompatible syntax and growing number of features which won't be > backported to 2.x. > > Suggestions: > > - We need a public discussion about possible date for dropping Python > 2 support. > - Early 2018 should give enough time for a graceful transition. > > -- > Best, > Maciej > >
