A minor point but often when onboarding, folks will try things verbatim from the website and documentation:
https://github.com/search?q=repo%3Aapache%2Fbeam+python3.7+lang%3AMarkdown+&type=code Granted, the most popular combo there was not present in this search, so it's probably not terribly significant, compared to the reason Robert is guessing. Dunno what we can do about that without going all out in specifying templated versions to use in our various docs. (That has the different problem of ensuring everything being described actually works as typed out, and we are not set up to efficiently validate that for every release.) On 2024/08/26 17:30:23 Robert Bradshaw via dev wrote: > So, 3.8 remains the most popular python version per pypi: > https://pypistats.org/packages/apache-beam > > Breaking down by Beam version over the last 90 days we get > > https://docs.google.com/spreadsheets/d/1-PPxZHs17aXvXgdl439tF7IqIs0XUxtDbDxGYcBg92I > > Which shows that this remains true even for the latest Beam releases. > (Interestingly, one of the most popular combinations is the Python 3.7 + > Beam 2.48. I wonder if people are holding off upgrading Beam due to Python > 3.7 being dropped...) > > Of course, the relationship between pypi downloads and actual customer > usage is not 1:1, but is likely directional at least. > > On the other hand, I do like the idea of letting the Python EoL cycle drive > our own supported versions. Given that 3.8 EoL is in October and our > release is (hopefully) also in October, what if instead we planned on > making 2.60 (tentatively) the last officially supported 3.8 release instead > of the release in which we drop 3.8 and then see what the stats say once > Python is officially EoL. Yes, we could just drop it if that's the > consensus, but given these usage numbers I don't think the case is so clear > cut. > > We could also look at what our dependencies are doing. And if supporting > 3.8 becomes difficult (e.g. is it being removed from github actions?) > that's another reason to do so. > > > [image: Skärmavbild 2024-08-26 kl. 10.08.09 fm.png] > > > > On Mon, Aug 26, 2024 at 9:42 AM Robert Burke <rob...@frantil.com> wrote: > > > I'd take care only relying on the most recent release (as much as it > > supports the consensus point). The most recent beam version is inherently > > going to have smaller and less consistent numbers, vs N-1 or N-2, since > > only the most keen or in need updates immediately. > > > > On Mon, Aug 26, 2024, 9:27 AM Danny McCormick via dev <dev@beam.apache.org> > > wrote: > > > >> Was about to respond, Rebo you beat me to it! I agree DockerHub is the > >> right thing to look at since Pypi reporting isn't awesome, I think we > >> should only look at the most recent versions though, since 3.8 will work > >> for old versions forever. > >> > >> For 2.58.0 last month (partial month results), I see: > >> > >> "Repo","Unique IPs","Pull by tag","Pull by digest","Version check" > >> "beam_python312_sdk",151,70,0,410 > >> "beam_python311_sdk",151,64,0,360 > >> "beam_python310_sdk",40,97,0,13 > >> "beam_python3.9_sdk",18,388,0,14 > >> "beam_python3.8_sdk",36,97,0,2 > >> > >> So it was <10% of pulls (including our automation as Rebo pointed out) > >> > >> I'll join Jack, Kenn, and Rebo and agree dropping support is the right > >> thing here. The plan SGTM as well. > >> > >> Thanks, > >> Danny > >> > >> On Mon, Aug 26, 2024 at 5:21 PM Robert Burke <rob...@frantil.com> wrote: > >> > >>> As an approximation we can use the docker container pulls at least. > >>> > >>> > >>> Py version : Pulls last week > >>> > >>> 3.8: 7476 > >>> 3.9: 1,259 > >>> 3.10: 6169 > >>> 3.11: 2999 > >>> 3.12: 241 > >>> > >>> 3.7: 395 > >>> 3.6: 241 > >>> 3.4: 156 > >>> 2.7: 188 > >>> > >>> But note that any of our automation for 3.8 that pulls containers would > >>> impact these result too. > >>> > >>> I will note that Beam dropping 3.8 support shouldn't be a problem given > >>> the general end of support of 3.8. > >>> > >>> Users can always upgrade their python version separately from the Beam > >>> version, and then update the Beam version. Ultimately, the cost of the > >>> latest and greatest version, is staying up to date. > >>> > >>> > >>> On Mon, Aug 26, 2024, 8:24 AM Kenneth Knowles <k...@apache.org> wrote: > >>> > >>>> SGTM > >>>> > >>>> Incidentally I poked around on pypi for a minute but didn't find even > >>>> basic download analytics. Do we have data about usage of Python versions? > >>>> (this is not pushback - I'm all for turning things down on a natural pace > >>>> (or faster!); I'm just even *more* for having data around it) > >>>> > >>>> Kenn > >>>> > >>>> On Mon, Aug 26, 2024 at 10:59 AM Jack McCluskey via dev < > >>>> dev@beam.apache.org> wrote: > >>>> > >>>>> Hey everyone, > >>>>> > >>>>> With Python 3.8 reaching end-of-life in October, I've started the work > >>>>> of removing support in the Beam repository. The aim is to target Beam > >>>>> release 2.60.0 for this, since the expected release cut date is on > >>>>> October 2nd, 2024. The start of this effort is at > >>>>> https://github.com/apache/beam/pull/32283/, updating our GitHub > >>>>> Actions workflows. For many workflows like our unit test suites this is > >>>>> not > >>>>> a large change; the Python version matrix simply omits 3.8 and runs on > >>>>> the > >>>>> remaining python versions as expected. This is more complicated for a > >>>>> number of workflows that currently only run on 3.8 or both 3.8 and > >>>>> 3.12, as > >>>>> GitHub will not run the updated actions in the main repository until > >>>>> the PR > >>>>> updating them is submitted. This can already be seen in some workflow > >>>>> runs > >>>>> on the PR where Python 3.8 is no longer being installed in the runner > >>>>> environment, leading to failures. > >>>>> > >>>>> The current plan is to do as much validation of the new workflow files > >>>>> as I can before the above PR is submitted (hopefully the week after Beam > >>>>> Summit,) then focus on getting any potential workflow breakages resolved > >>>>> before removing the core Python 3.8 support from the package. There may > >>>>> be > >>>>> some instability with our workflows, and I will try my best to resolve > >>>>> things as they pop up. This is the first Python version to have support > >>>>> dropped since we migrated to GitHub Actions, so there's going to be a > >>>>> decent amount of trial and error as we navigate this. That said, if you > >>>>> notice problems please let me know! Either file a standalone issue and > >>>>> tag > >>>>> me on it (@jrmccluskey) or leave a comment on > >>>>> https://github.com/apache/beam/issues/31192 so I can take a look. > >>>>> > >>>>> Thanks, > >>>>> > >>>>> Jack McCluskey > >>>>> > >>>>> -- > >>>>> > >>>>> > >>>>> Jack McCluskey > >>>>> SWE - DataPLS PLAT/ Dataflow ML > >>>>> RDU > >>>>> jrmcclus...@google.com > >>>>> > >>>>> > >>>>> >
