Re: Replace uses of deprecated Python module distutils.sysconfig

On 2022-02-01 18:37:02 -0500, Tom Lane wrote: > So I'm going to go ahead with patching; but does anyone want > to change their vote? (We can always "git revert".) +1 for going ahead

Re: Replace uses of deprecated Python module distutils.sysconfig

Noah Misch writes: > On Mon, Jan 31, 2022 at 05:18:47PM -0500, Tom Lane wrote: >> If nobody else has weighed in by tomorrow, I'll backpatch to v10. > Works for me. I agree wanting Python 3.12 w/ PG10.latest is far more likely > than wanting Python 2.6 or 3.1. If someone lodges a non-academic co

Re: Replace uses of deprecated Python module distutils.sysconfig

On Mon, Jan 31, 2022 at 05:18:47PM -0500, Tom Lane wrote: > [ I was hoping for more opinions, but I guess nobody cares but us ] > > Andres Freund writes: > > On 2022-01-27 17:53:02 -0500, Tom Lane wrote: > >> So now we need to discuss whether we want to back-patch this. > >> Pros: avoid configure

Re: Replace uses of deprecated Python module distutils.sysconfig

[ I was hoping for more opinions, but I guess nobody cares but us ] Andres Freund writes: > On 2022-01-27 17:53:02 -0500, Tom Lane wrote: >> So now we need to discuss whether we want to back-patch this. >> Pros: avoid configure warning now (not worth much); avoid outright >> build failure on Pyth

Re: Replace uses of deprecated Python module distutils.sysconfig

Hi, On 2022-01-27 17:53:02 -0500, Tom Lane wrote: > Andres Freund writes: > > Thanks! Looks pretty good so far. Including on machines that were broken in > > take 1... > > Just about all of the buildfarm has reported in now, and it's all good. > So now we need to discuss whether we want to back-p

Re: Replace uses of deprecated Python module distutils.sysconfig

Andres Freund writes: > Thanks! Looks pretty good so far. Including on machines that were broken in > take 1... Just about all of the buildfarm has reported in now, and it's all good. So now we need to discuss whether we want to back-patch this. Pros: avoid configure warning now (not worth much)

Re: Replace uses of deprecated Python module distutils.sysconfig

Hi, Sorry I somehow missed that email. On Mon, Jan 24, 2022 at 7:53 AM Tom Lane wrote: > > Now, most of those are BSD machines --- but lapwing isn't. > It says > > checking for python... (cached) /usr/bin/python > configure: using python 3.6.9 (default, Jan 14 2022, 06:45:55) > checking for Pyth

Re: Replace uses of deprecated Python module distutils.sysconfig

On 2022-01-25 18:33:55 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2022-01-25 12:45:15 -0500, Tom Lane wrote: > >> I guess next steps are to revert f032f63e7 and then retry e0e567a10 > >> with that change. Who's going to do the honors? > > > I assume Peter is done working for the day.

Re: Replace uses of deprecated Python module distutils.sysconfig

Andres Freund writes: > On 2022-01-25 12:45:15 -0500, Tom Lane wrote: >> I guess next steps are to revert f032f63e7 and then retry e0e567a10 >> with that change. Who's going to do the honors? > I assume Peter is done working for the day. I'm stuck in meetings and stuff > for another 2-3 hours.

Re: Replace uses of deprecated Python module distutils.sysconfig

Hi, On 2022-01-25 12:45:15 -0500, Tom Lane wrote: > As of now, 92 buildfarm animals have reported results from f032f63e7. > Every single one of them reports that all the different methods you > tested give the same answer. So it looks to me like we should just > go with get_config_var('INCLUDEPY'

Re: Replace uses of deprecated Python module distutils.sysconfig

I wrote: > It's a little bit too soon to decide that INCLUDEPY is reliably equal > to that, but if it still looks that way tomorrow, I'll be satisfied. As of now, 92 buildfarm animals have reported results from f032f63e7. Every single one of them reports that all the different methods you tested g

Re: Replace uses of deprecated Python module distutils.sysconfig

I wrote: > Yeah, I was just noticing that. It looks like the whole business > with checking both get_python_inc(False) and get_python_inc(True) > has been useless from the start: none of the buildfarm animals report > more than one -I switch in "checking Python include directories". Also, that ap

Re: Replace uses of deprecated Python module distutils.sysconfig

Andres Freund writes: > ... But none of the systems report a get_python_inc(False) differing from > get_python_inc(True), or from the value of INCLUDEPY. So I don't see a reason > for why it'd not? Yeah, I was just noticing that. It looks like the whole business with checking both get_python_inc

Re: Replace uses of deprecated Python module distutils.sysconfig

Hi, On 2022-01-23 19:49:57 -0800, Andres Freund wrote: > > To avoid too noisy breakages, we could have python.m4 emit INCLUDEPY and > > then > > search the bf logs in a day or three? > > Maybe something like the attached? Not particularly nice, but should give us > most of the relevant informati

Re: Replace uses of deprecated Python module distutils.sysconfig

Peter Eisentraut writes: > Also note that python-config is itself a Python script that uses > sysconfig and includes code like this: > elif opt in ('--includes', '--cflags'): > flags = ['-I' + sysconfig.get_path('include'), > '-I' + sysconfig.get_path('platinclude

Re: Replace uses of deprecated Python module distutils.sysconfig

Hi, On 2022-01-24 21:48:21 +0100, Peter Eisentraut wrote: > Also note that python-config is itself a Python script that uses sysconfig > and includes code like this: Huh. It's a shell script on my debian system. Looks like the python source tree has both. Not sure what / who decides which is used

Re: Replace uses of deprecated Python module distutils.sysconfig

On 24.01.22 03:53, Andres Freund wrote: On 2022-01-23 21:31:52 -0500, Tom Lane wrote: Andres Freund writes: No, not really. There generally seems to be very little documentation about what one is supposed to use when embedding python (rather than building a python module). The only thing I rea

Re: Replace uses of deprecated Python module distutils.sysconfig

Hi, On 2022-01-23 18:53:01 -0800, Andres Freund wrote: > I don't really understand what the various "platform" variables / paths are > supposed to do. The code says: > def get_python_inc(plat_specific=0, prefix=None): > """Return the directory containing installed Python header files. > >

Re: Replace uses of deprecated Python module distutils.sysconfig

Andres Freund writes: > To avoid too noisy breakages, we could have python.m4 emit INCLUDEPY and then > search the bf logs in a day or three? +1, it'd give us some info without breaking the farm. regards, tom lane

Re: Replace uses of deprecated Python module distutils.sysconfig

Hi, On 2022-01-23 21:31:52 -0500, Tom Lane wrote: > Andres Freund writes: > > No, not really. There generally seems to be very little documentation about > > what one is supposed to use when embedding python (rather than building a > > python module). The only thing I really see is: > > > https:/

Re: Replace uses of deprecated Python module distutils.sysconfig

Andres Freund writes: > No, not really. There generally seems to be very little documentation about > what one is supposed to use when embedding python (rather than building a > python module). The only thing I really see is: > https://docs.python.org/3/extending/embedding.html#compiling-and-link

Re: Replace uses of deprecated Python module distutils.sysconfig

Hi, On 2022-01-23 20:50:23 -0500, Tom Lane wrote: > Andres Freund writes: > > I think this might be problem on our own end, actually. The > > distutils.sysconfig > > code did > > a = '-I' + distutils.sysconfig.get_python_inc(False) > > b = '-I' + distutils.sysconfig.get_python_inc(True) > > whic

Re: Replace uses of deprecated Python module distutils.sysconfig

Andres Freund writes: > I think this might be problem on our own end, actually. The > distutils.sysconfig > code did > a = '-I' + distutils.sysconfig.get_python_inc(False) > b = '-I' + distutils.sysconfig.get_python_inc(True) > which the patch upthread changed to > +a = '-I' + sysconfig.get_path

Re: Replace uses of deprecated Python module distutils.sysconfig

Hi, On 2022-01-23 19:00:41 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2022-01-23 18:31:44 -0500, Tom Lane wrote: > >> Yeah, but we don't know whether there are any versions of the Debian > >> packaging in which they fixed the file layout, so that 'posix_local' > >> actually does descri

Re: Replace uses of deprecated Python module distutils.sysconfig

Andres Freund writes: > On 2022-01-23 18:31:44 -0500, Tom Lane wrote: >> Yeah, but we don't know whether there are any versions of the Debian >> packaging in which they fixed the file layout, so that 'posix_local' >> actually does describe the layout. > I think posix_local try to achieve somethin

Re: Replace uses of deprecated Python module distutils.sysconfig

I wrote: > Yeah, but we don't know whether there are any versions of the Debian > packaging in which they fixed the file layout, so that 'posix_local' > actually does describe the layout. Actually ... scraping the buildfarm to see what we're currently finding shows that the following machines are

Re: Replace uses of deprecated Python module distutils.sysconfig

Hi, On 2022-01-23 18:31:44 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2022-01-23 18:11:41 -0500, Tom Lane wrote: > >> Anyway, trying to figure out whether we're on a Debian package with this > >> mistake doesn't seem any cleaner than what I proposed. (In particular, > >> blindly chang

Re: Replace uses of deprecated Python module distutils.sysconfig

Andres Freund writes: > On 2022-01-23 18:11:41 -0500, Tom Lane wrote: >> Anyway, trying to figure out whether we're on a Debian package with this >> mistake doesn't seem any cleaner than what I proposed. (In particular, >> blindly changing to a different scheme without a check to see what's reall

Re: Replace uses of deprecated Python module distutils.sysconfig

Andres Freund writes: > The relevant part of distutils-install-layout.diff explaining this is: > +(0) > + Starting with Python-2.6 Debian/Ubuntu uses for the Python which comes > within > + the Linux distribution a non-default name for the installation directory. > This > + is to avoid ov

Re: Replace uses of deprecated Python module distutils.sysconfig

Hi, On 2022-01-23 18:11:41 -0500, Tom Lane wrote: > It may be that one of the other "scheme" values accurately describes > Debian's actual layout of this package. I didn't check, because the > scheme is defined to be platform-specific. posix_prefix does, as far as I can see. > Specifying a par

Re: Replace uses of deprecated Python module distutils.sysconfig

Andres Freund writes: > On 2022-01-23 16:06:21 -0500, Tom Lane wrote: >> +'posix_local': { >> +'stdlib': '{base}/lib/python{py_version_short}', >> +'platstdlib': '{platbase}/lib/python{py_version_short}', >> +'purelib': >> '{base}/local/lib/python{py_version_short}/dis

Re: Replace uses of deprecated Python module distutils.sysconfig

Hi, On 2022-01-23 16:06:21 -0500, Tom Lane wrote: > (I don't know > enough about Debian packaging to find the smoking gun though; > what apt-get claims is the source package contains no trace of > this diff.) There's no sign of comparable changes in > /usr/lib/python3.5/sysconfig.py on the same m

Re: Replace uses of deprecated Python module distutils.sysconfig

Hi, On 2022-01-23 16:06:21 -0500, Tom Lane wrote: > I wrote: > > Based on the buildfarm results so far, the problem can be described > > as "some installations say /usr/local when they should have said /usr". > > I experimented with the attached delta patch and it fixes the problem > > on my Debia

Re: Replace uses of deprecated Python module distutils.sysconfig

I wrote: > Based on the buildfarm results so far, the problem can be described > as "some installations say /usr/local when they should have said /usr". > I experimented with the attached delta patch and it fixes the problem > on my Debian 9 image. (I don't know Python, so there may be a better >

Re: Replace uses of deprecated Python module distutils.sysconfig

I wrote: > The early returns are not great: we have about half a dozen machines > so far that are finding python3, and reporting sane-looking Python > include paths, but not finding Python.h. They're all Linux-oid > machines, so I suppose what is going on is that they have the base > python3 packa

Re: Replace uses of deprecated Python module distutils.sysconfig

On 1/18/22 19:18, Tom Lane wrote: > I wrote: >> Are we sure it's an issue within Python, rather than something we >> could dodge by invoking sysconfig differently? It's hard to believe >> that sysconfig could be totally unfit for the purpose of finding out >> the include path and would remain so

Re: Replace uses of deprecated Python module distutils.sysconfig

=?UTF-8?Q?Juan_Jos=C3=A9_Santamar=C3=ADa_Flecha?= writes: > On Wed, Jan 19, 2022 at 9:40 PM Tom Lane wrote: >> Done. (I couldn't find any equivalent logic in the MSVC build scripts >> though; is there something I missed?) > MSVC will use the path configured in src\tools\msvc\config.pl > $conf

Re: Replace uses of deprecated Python module distutils.sysconfig

I wrote: > Peter Eisentraut writes: >> On 19.01.22 01:18, Tom Lane wrote: >>> ... I'm tempted to resurrect the >>> idea of changing configure's probe order to "python3 python python2" >>> in the meantime, just so we can see how much of the buildfarm is ready >>> for that. >> This seems sensible i

Re: Replace uses of deprecated Python module distutils.sysconfig

On Wed, Jan 19, 2022 at 9:40 PM Tom Lane wrote: > > Done. (I couldn't find any equivalent logic in the MSVC build scripts > though; is there something I missed?) > > MSVC will use the path configured in src\tools\msvc\config.pl > $config->{"python"}, there is no ambiguity. Regards, Juan José

Re: Replace uses of deprecated Python module distutils.sysconfig

Peter Eisentraut writes: > On 19.01.22 01:18, Tom Lane wrote: >> ... I'm tempted to resurrect the >> idea of changing configure's probe order to "python3 python python2" >> in the meantime, just so we can see how much of the buildfarm is ready >> for that. > This seems sensible in any case, given

Re: Replace uses of deprecated Python module distutils.sysconfig

On 19.01.22 01:18, Tom Lane wrote: Anyway, based on these results, we might have better luck switching to sysconfig after we start forcing python3. I'm tempted to resurrect the idea of changing configure's probe order to "python3 python python2" in the meantime, just so we can see how much of th

Re: Replace uses of deprecated Python module distutils.sysconfig

I wrote: > Anyway, based on these results, we might have better luck switching to > sysconfig after we start forcing python3. On the other hand, that answer is not back-patchable, and we surely need a back-patchable fix, because people will try to build the back branches against newer pythons. Ba

Re: Replace uses of deprecated Python module distutils.sysconfig

I wrote: > Are we sure it's an issue within Python, rather than something we > could dodge by invoking sysconfig differently? It's hard to believe > that sysconfig could be totally unfit for the purpose of finding out > the include path and would remain so for multiple years. I dug up a Debian 9

Re: Replace uses of deprecated Python module distutils.sysconfig

Peter Eisentraut writes: > I don't have a clear idea how to fix this in the long run. We would > perhaps need to determine at which points the various platforms had > fixed this issue in their Python installations and select between the > old and the new approach based on that. Seems messy.

Re: Replace uses of deprecated Python module distutils.sysconfig

On 18.01.22 16:24, Tom Lane wrote: Peter Eisentraut writes: Also, considering the failure on prairiedog, I do see now on that the sysconfig module is "New in version 3.2". I had interpreted the fact that it exists in version 2.7 that that incl

Re: Replace uses of deprecated Python module distutils.sysconfig

Peter Eisentraut writes: > Also, considering the failure on prairiedog, I do see now on > that the sysconfig > module is "New in version 3.2". I had interpreted the fact that it > exists in version 2.7 that that includes all higher versions, b

Re: Replace uses of deprecated Python module distutils.sysconfig

On 09.12.21 14:31, Tom Lane wrote: Peter Eisentraut writes: On 02.12.21 08:20, Peter Eisentraut wrote: Buildfarm impact: gaur and prariedog use Python 2.6 and would need to be upgraded. Tom, are you planning to update the Python version on these build farm members? I realize these are very

Re: Replace uses of deprecated Python module distutils.sysconfig

Peter Eisentraut writes: > On 02.12.21 08:20, Peter Eisentraut wrote: >> Buildfarm impact: >> gaur and prariedog use Python 2.6 and would need to be upgraded. > Tom, are you planning to update the Python version on these build farm > members? I realize these are very slow machines and this migh

Re: Replace uses of deprecated Python module distutils.sysconfig

On 02.12.21 08:20, Peter Eisentraut wrote: Buildfarm impact: gaur and prariedog use Python 2.6 and would need to be upgraded. Tom, are you planning to update the Python version on these build farm members? I realize these are very slow machines and this might take some time; I'm just wonder

Re: Replace uses of deprecated Python module distutils.sysconfig

On 02.12.21 19:22, Tom Lane wrote: That's surely no problem in HEAD, but as you say, it is an issue for the older branches. How difficult would it be to teach configure to try both ways, or adapt based on its python version check? I think it wouldn't be unreasonable to do that. I'll look into

Re: Replace uses of deprecated Python module distutils.sysconfig

Peter Eisentraut writes: > With Python 3.10, configure spits out warnings about the module > distutils.sysconfig being deprecated and scheduled for removal in Python > 3.12: Bleah. > This patch changes the uses in configure to use the module sysconfig > instead. The logic stays the same. (I

Replace uses of deprecated Python module distutils.sysconfig

.)From fcfd4f1702f0cef7d0f6564aff94c4a6258f24e6 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Thu, 2 Dec 2021 08:08:44 +0100 Subject: [PATCH] Replace uses of deprecated Python module distutils.sysconfig With Python 3.10, configure spits out warnings about the module distutils.sysconfig be