For the reference I raised question of Python 2 support before - http://apache-spark-developers-list.1001551.n3.nabble.com/Future-of-the-Python-2-support-td20094.html
On Sat, 15 Sep 2018 at 15:14, Alexander Shorin <kxe...@apache.org> wrote: > What's the release due for Apache Spark 3.0? Will it be tomorrow or > somewhere at the middle of 2019 year? > > I think we shouldn't care much about Python 2.x today, since quite > soon it support turns into pumpkin. For today's projects I hope nobody > takes into account support of 2.7 unless there is some legacy still to > carry on, but do we want to take that baggage into Apache Spark 3.x > era? The next time you may drop it would be only 4.0 release because > of breaking change. > > -- > ,,,^..^,,, > On Sat, Sep 15, 2018 at 2:21 PM Maciej Szymkiewicz > <mszymkiew...@gmail.com> wrote: > > > > There is no need to ditch Python 2. There are basically two options > > > > Use stub files and limit yourself to support only Python 3 support. > Python 3 users benefit from type hints, Python 2 users don't, but no core > functionality is affected. This is the approach I've used with > https://github.com/zero323/pyspark-stubs/. > > Use comment based inline syntax or stub files and don't use backward > incompatible features (primarily typing module - > https://docs.python.org/3/library/typing.html). Both Python 2 and 3 is > supported, but more advanced components are not. Small win for Python 2 > users, moderate loss for Python 3 users. > > > > > > > > On Sat, 15 Sep 2018 at 02:38, Nicholas Chammas < > nicholas.cham...@gmail.com> wrote: > >> > >> Do we need to ditch Python 2 support to provide type hints? I don’t > think so. > >> > >> Python lets you specify typing stubs that provide the same benefit > without forcing Python 3. > >> > >> 2018년 9월 14일 (금) 오후 8:01, Holden Karau <hol...@pigscanfly.ca>님이 작성: > >>> > >>> > >>> > >>> On Fri, Sep 14, 2018, 3:26 PM Erik Erlandson <eerla...@redhat.com> > wrote: > >>>> > >>>> To be clear, is this about "python-friendly API" or "friendly python > API" ? > >>> > >>> Well what would you consider to be different between those two > statements? I think it would be good to be a bit more explicit, but I don't > think we should necessarily limit ourselves. > >>>> > >>>> > >>>> On the python side, it might be nice to take advantage of static > typing. Requires python 3.6 but with python 2 going EOL, a spark-3.0 might > be a good opportunity to jump the python-3-only train. > >>> > >>> I think we can make types sort of work without ditching 2 (the types > only would work in 3 but it would still function in 2). Ditching 2 entirely > would be a big thing to consider, I honestly hadn't been considering that > but it could be from just spending so much time maintaining a 2/3 code > base. I'd suggest reaching out to to user@ before making that kind of > change. > >>>> > >>>> > >>>> On Fri, Sep 14, 2018 at 12:15 PM, Holden Karau <hol...@pigscanfly.ca> > wrote: > >>>>> > >>>>> Since we're talking about Spark 3.0 in the near future (and since > some recent conversation on a proposed change reminded me) I wanted to open > up the floor and see if folks have any ideas on how we could make a more > Python friendly API for 3.0? I'm planning on taking some time to look at > other systems in the solution space and see what we might want to learn > from them but I'd love to hear what other folks are thinking too. > >>>>> > >>>>> -- > >>>>> Twitter: https://twitter.com/holdenkarau > >>>>> Books (Learning Spark, High Performance Spark, etc.): > https://amzn.to/2MaRAG9 > >>>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau > >>>> > >>>> > > > > >
