> I have the impression that NEP 29 tried to very rigidly define end of life by a specific timeline with no flexibility for the potential that the official Python release timelines are not rigid and fixed in stone forever
The PR linked above mentions quite often that they do take variations/changes of the Python release schedule into account. For example, NEP 29 writes " support at least all minor versions of Python introduced and released in the prior 42 months *from the anticipated release date* with a minimum of 2 minor versions of Python". With a rigid schedule, this is a unnecessary duplication - but the "at least two minor versions" is there to prevent the worst case scenario where python releases are delayed by a few years and no python version would be supported otherwise. But yes, naturally there are some included expectations about the Python release schedule included in NEP 29 and it would probably be revised if Python changes its schedule. Conversely, NEP 29 is adapted by so many libraries that Python itself includes it in discussions about its release schedule (e.g. https://peps.python.org/pep-0605/#implications-for-the-proposed-scientific-python-ecosystem-support-period) On Saturday, 27 May 2023 at 00:56:12 UTC+8 Tobias Diez wrote: > Here is the PR that introduced NEP 29: > https://github.com/numpy/numpy/pull/14086. The main discussion happened > in person at scipy 2019 and in webcalls. But a lot of the points raised > here are answered in this PR. > > For example, the point about Linux LTS / Python EOL is addressed by one of > the writers of NEP 29 as follows ( > https://github.com/numpy/numpy/pull/14086#issuecomment-516661140) > " > > Conversely, I am not sure of a real world example where a user must have: > > - 4 year old version of Python > - just-released version of numpy / Matplotilb / scikit-learn [add sage > here] > > If you are making the choice to stay on an old version of Python (which > there are good reasons to do!) then why would you not *also* make the > conservative choice to stay on the contemporaneous version of the scipy > stack? > " > > The point that "NEP is about libraries and not downstream applications" is > also not valid since this clearly was part of the discussion back then. For > example, one of the core maintainers of jupyter and ipython writes > https://github.com/numpy/numpy/pull/14086#issuecomment-516904953 > " There is also the same catch 22 than dropping python 2; downstream was > not dropping because upstream was still supporting Python 2, and upstream > was still supporting Python 2 because downstream was relying on it." > In other words, if all downstream applications would have a more > conservative policy, then naturally also numpy etc would need to adapt. NEP > 29 is about homogenizing the supported Python version in the scientific > python community. This is also very clearly written in the first line of > the NEP: "recommends that *all projects *across the Scientific Python > ecosystem adopt a common “time window-based” policy for support of Python > and NumPy versions" (emphasize mine). Its "all projects", not "all upstream > libraries". So unless someone wants to argue that sage is not part of the > scientific python ecosystem, we are directly addressed. > > By the way, what happened to the "please let the discussion go somewhere > else"? Should I restart the voting in a new thread? > On Friday, 26 May 2023 at 23:10:52 UTC+8 William Stein wrote: > >> Hi, >> >> To help with people who want to make an informed decision, is there >> any public discussion of the original NEP 29 proposal? >> >> The only thing I could find was this post from Sebastian Berg, where he >> says at >> >> >> https://mail.python.org/pipermail/numpy-discussion/2019-October/080128.html >> >> "We propose formally accepting the NumPy enhancement proposal 29... If >> there are no objections within a week it may be accepted.". >> There's no public voting or anything, and there's exactly one quick >> random response of "yes" in the thread. >> >> In the actual NEP 29 proposal >> https://numpy.org/neps/nep-0029-deprecation_policy.html there is a >> section labeled "Discussion" and it is empty. >> I would love to read about the discussions for/against NEP 29 from >> when they decided on it in the first place! >> >> There are follow up discussions, just like this one, by other >> projects, e.g., for sympy here: >> https://github.com/sympy/sympy/issues/21884, which >> is still not decided, and has been under regular discussion for nearly >> 2 years. There are many other similar conversations. But they are >> all about whether to align with numpy or not, and the core question of >> why it is best to ignore what the official upstream python supports >> isn't as relevant. >> >> The original NEP 29 says "The Python release cadence increased in PEP >> 0602, [...] Thus, we do not expect our users to upgrade Python faster, >> and our 42 month support window will cover the same portion of the >> upstream support of any given Python release." I don't really know >> what that means, but I have the impression that NEP 29 tried to very >> rigidly define end of life by a specific timeline with no flexibility >> for the potential that the official Python release timelines are not >> rigid and fixed in stone forever, while simultaneously acknowledging >> this conflict. I would love to see the arguments for doing that, >> which could be compelling. I fully realize that this is something >> that came from maintainers of open source software, and they were >> probably feeling annoyed and burned out, and this NEP 29 may have >> helped them keep their sanity. But if that's the case, it's decided >> in secret as far as I can tell. >> >> -- William >> >> On Fri, May 26, 2023 at 6:34 AM Matthias Koeppe >> <matthia...@gmail.com> wrote: >> > >> > Thanks, Tobias, for opening this vote thread. Here on sage-devel, this >> is a much better setting than what you attempted in >> https://github.com/sagemath/sage/pull/35404#issuecomment-1504474945 >> > >> > I am voting NO. >> > >> > There's a simple rationale: >> > >> > I. This proposed policy change does not solve any problem. There are no >> problems whatsoever with how we have managed the support of Python versions >> since 2020 (when it became possible to use system Python instead of only >> the Python from our SPKG.) >> > >> > II. The proposed policy change creates new problems. Following this >> policy would force us to drop support for a particular Python version at >> times when it would be harmful for our project. Specifically, right now it >> would *force* us to drop support for Python 3.8 and hence for using the >> default Python on Ubuntu Linux 20.04 (an LTS release, with "End of Standard >> Support" April 2025 and "End Of Life" April 2030. It is the very point of >> LTS releases to provide a stable software environment; so, yes, Python 3.8 >> is fully supported, and if Python 3.8.x had bugs relevant for Sage, we >> would know about it because we are testing. >> > >> > III. Our existing practice is to carefully consider and weigh various >> factors that are relevant for the Sage project, rather than following a >> fixed schedule that is set by an external, largely separate community. I >> briefly explained what we do in >> https://groups.google.com/g/sage-devel/c/j1cwbTU8aOU/m/x3qOdPB5BQAJ; but >> I'll expand here on some important factors: >> > >> > a) Sage has a dual role as a library ("project") and as a distribution. >> NEP 29 was designed for projects, and not for software distributions. >> > >> > b) In Sage, we only have one line of releases. Hence users who want any >> bug fixes need to use our latest version. In contrast, just like Python >> itself, many other projects have at least two separate branches: A branch >> on which the cutting edge development takes place (new features etc.), and >> a branch from which maintenance updates are made. For example, NumPy >> removed support of Python 3.8 in their development branch earlier this >> year; but this in preparation for the 1.25 release expected this summer. >> NumPy continued to make maintenance releases on the 1.24 branch ( >> https://github.com/numpy/numpy/releases), and by policy, these >> maintenance upgrades never drop the support of a previously supported >> version. >> > >> > c) NEP29 was designed for and is in use by a part of the scientific >> Python community, to address the need to be able to use features of new >> Python versions and features of NumPy/SciPy faster. This is important for >> many projects that have NumPy/SciPy as their dependencies. >> > >> > d) In contrast, our uses of NumPy/SciPy in the Sage library are very >> basic and dating back by about a decade; with the exception of the optional >> use of a recent SciPy feature (the high-performance optimization solver >> HiGHS, see >> https://github.com/sagemath/sage/wiki/Sage-10.0-Release-Tour#linear-programming-and-extensions), >> >> which motivated our quick upgrade to the current SciPy version in Sage. And >> as another example, also our use of matplotlib in the library dates back by >> a decade or more; we regularly have to update when new matplotlib versions >> come out that make API changes, but we haven't picked up any new features >> in a very long time. >> > >> > e) Yes, synchronization between projects matters for maintainability. >> But Sage is downstream of lots of Python packages; before we can offer >> support for a new version of Python, we often have to wait until all or >> most of our dependencies provide support for that new version. For example, >> some projects are actively working on support for the Python 3.12 release >> expected this early Fall; but for us, this is not actionable because we >> have to wait for critical dependencies; see >> https://github.com/sagemath/sage/issues/34788 . Likewise, it is not >> useful for us to drop support for an old version before there is a clear >> benefit for us, brought for example by important upgrades that have dropped >> support already. >> > >> > >> > (As suggested by Tobias, any discussion of my explanations above is >> best sent to the >> https://groups.google.com/g/sage-devel/c/j1cwbTU8aOU/m/2sTiwdKPBQAJ >> thread.) >> > >> > >> > On Friday, May 26, 2023 at 3:12:16 AM UTC-7 Tobias Diez wrote: >> >> >> >> Dear Sage developers, >> >> >> >> the NumPy enhancement proposal 29: "Recommend Python and Numpy version >> support as a community policy standard" (available at >> https://numpy.org/neps/nep-0029-deprecation_policy.html) specifies when >> it's okay to drop support for old Python version. >> >> >> >> Namely, a release should support "all minor versions of Python >> released 42 months prior to the project, and at minimum the two latest >> minor versions. ". In particular, this means: >> >> - Currently, Sage should support > 3.8. >> >> - On Apr 05, 2024 we should drop support for Python 3.9 (initially >> released on Oct 05, 2020) >> >> >> >> Based on previous discussions on this topic ( >> https://groups.google.com/g/sage-devel/c/j1cwbTU8aOU/m/2sTiwdKPBQAJ, >> https://github.com/sagemath/sage/issues/30384, >> https://github.com/sagemath/sage/pull/35403), I'm calling for a vote on >> adapting the Python version support of NEP 29 in Sage. Voting ends at the >> 7th June, AoE. Please use this thread only for sending votes, to make it >> easier to count them afterward; and use the thread >> https://groups.google.com/g/sage-devel/c/j1cwbTU8aOU/m/2sTiwdKPBQAJ for >> discussion. >> >> >> >> Summary of the points brought forward in the discussions linked above >> >> 1. The current practice in Sage is to evaluate whether to increase the >> minimum version of Python supported at the beginning of each release cycle. >> With regard to such a practice, the NEP 29 documents remarks "As there is >> no objective threshold to when the minimum version should be dropped, it is >> easy for these version support discussions to devolve into bike shedding >> and acrimony." Sadly, an example of this can be found in the current >> discussion of dropping Python 3.8 support in >> https://github.com/sagemath/sage/pull/35404 with emotions running so >> high that sage-abuse had to step in. Adopting a version policy would >> prevent such discussions. On the other hand, by following a given policy, >> we would loose some flexibility. >> >> 2. The main idea of NEP 29 is to have a community-wide standard. It is >> followed by many scientific packages such as Scipy, Matplotlib, IPython, >> Jupyter, Pandas, scikit, astropy, cuda, cirq, jax, pytorch among others. >> The adoption of NEP 29 will harmonize Sage's deprecation policy with these >> other major libraries. >> >> 3. The NEP 29 drop schedule is much faster than the EOL schedule of >> Python itself. Python 3.8 is supported until 2024-10, but NEP 29 already >> drops it 2023-04. However, adhering to the EOL schedule would prevent us to >> updating these packages that follow NEP 29. >> >> 4. The NEP 29 schedule is about one release cycle faster than the >> previous drops (e.g. Python 3.7 support was dropped in Sage 9.7 while >> according to NEP 29 it would have been Sage 9.6). >> >> 5. The faster drop schedule will free developer resources (less >> systems to test) and potentially increase developer productivity as it >> allows us to use newer language features. >> >> 6. The faster drop schedule might be inconvenient for users who rely >> on older Python versions. To some extend this is remedied by our python >> install package, and relatively straightforward upgrade paths on most >> system. One should also note that users relying on other scientific python >> packages are likely forced to upgrade anyway, since these other packages >> likely follow NEP 29. >> >> 7. The faster drop schedule would force users to upgrade to newer >> Python versions and thereby profit from fewer bugs and security issues. It >> is however questionable if Sage should play this educator role. >> >> 8. One of the main goals of NEP 29 is to improve downstream project >> planning by having a community-wide standard. This is currently not very >> relevant for us as Sage is currently upstream of nothing except for some >> user packages. With the modularization effort, this may change in the >> future. >> >> 9. There are not many other documented policies. As said above, most >> scientific python projects follow NEP 29. Most projects in web development >> (e.g flask) seem to drop a version once it reaches EOL. Machine learning >> projects follow a similar EOL policy (e.g. tensorflow) or roughly follow >> NEP 29 (scikit-learn). Some end-user applications have even stricter >> version constraints then NEP 29 (e.g. home-assistant only supports the two >> latest minor releases). >> >> >> > -- >> > You received this message because you are subscribed to the Google >> Groups "sage-devel" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an email to sage-devel+...@googlegroups.com. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/sage-devel/f6bf61cd-373f-4829-bfba-392ce32daeban%40googlegroups.com. >> >> >> >> >> >> -- >> William (http://wstein.org) >> > -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/9f9bce59-e1c1-474e-a37e-868d4eef22b8n%40googlegroups.com.
