Re: [Python-Dev] Remove embedded expat library?

2017-06-11 Thread Ronald Oussoren 

> On 9 Jun 2017, at 18:03, Ned Deily  wrote:
> 
> On Jun 9, 2017, at 08:43, Victor Stinner  wrote:
> 
>> I expect that all Linux distributions build Python using
>> --with-system-expat. It may become the default? What about macOS and
>> other operating systems?
> 
> The current default is --with-system-expat=no so, unless builders of Python 
> take explicit action, the bundled version of expat is used.  Using the 
> bundled version is also currently the case for the python.org macOS 
> installer, no idea what other distributors do.  Apple supplies a version of 
> expat with macOS so we presumably we could use the system version for the 
> installer.  Presumably (Zach?) we would need to continue to supply a version 
> of expat for Windows builds.  But do we need to for others?  If it were only 
> Windows, *then* perhaps it might make sense to make all the changes to move 
> expat out of cpython into the common repo for third-party Windows libs.

I don’t think it would be a good idea to rely on the system provided libexpat 
on macOS, as Apple is not exactly fast w.r.t. upgrading their external 
dependencies and could easily stop updating libraries when the no longer need 
them (see for example the mess w.r.t. OpenSSL).

Ronald
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove embedded expat library?

2017-06-11 Thread Victor Stinner 
Le 11 juin 2017 09:38, "Ronald Oussoren"  a écrit :

I don’t think it would be a good idea to rely on the system provided
libexpat on macOS, as Apple is not exactly fast w.r.t. upgrading their
external dependencies and could easily stop updating libraries when the no
longer need them (see for example the mess w.r.t. OpenSSL).



Ok, but can't we download expat instead of keeping an old copy in our
repisitory?

Having a copy is useful when we modify it. I don't that it is the case here.

Victor
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove embedded expat library?

2017-06-11 Thread Ronald Oussoren 

> On 11 Jun 2017, at 12:10, Victor Stinner  wrote:
> 
> Le 11 juin 2017 09:38, "Ronald Oussoren"  > a écrit :
> I don’t think it would be a good idea to rely on the system provided libexpat 
> on macOS, as Apple is not exactly fast w.r.t. upgrading their external 
> dependencies and could easily stop updating libraries when the no longer need 
> them (see for example the mess w.r.t. OpenSSL).
> 
> 
> Ok, but can't we download expat instead of keeping an old copy in our 
> repisitory?

Sure. The script that creates the installer already downloads a number of 
libraries, adding one more shouldn’t be a problem. 

> 
> Having a copy is useful when we modify it. I don't that it is the case here.

I don’t know why expat was included in the CPython tree and if those reasons 
are still valid. I therefore have no opinion on keeping it, other than that 
expat shouldn’t be kept in the CPython tree unless there’s a good reason for 
doing so. 

BTW. Removing 3th-party libraries from the source tree doesn’t fully isolate us 
from security issues in those libraries due to shipping the libraries in binary 
installers on Windows and macOS.  The removal does make maintenance easier (no 
need to guess whether or not there are local patches).

Ronald

> 
> Victor
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 538 (review round 2): Coercing the legacy C locale to a UTF-8 based locale

2017-06-11 Thread Martin (gzlist) via Python-Dev 
On 09/05/2017, Nick Coghlan  wrote:
>
> Enough changes have accumulated in PEP 538 since the start of the
> previous thread that it seems sensible to me to start a new thread
> specifically covering the current design (which aims to address all
> the concerns raised in the previous thread).
>
> I haven't requoted the PEP in full since it's so long, but will
> instead refer readers to the web version:
> https://www.python.org/dev/peps/pep-0538/

I did try to follow along via the mailing list threads, and have now
read over the PEP again. Responding now as I'm actually touching code
relevent to this again.

Broadly the proposal looks good to me. It does help one of the two
cases I care about, and does no serious harm.

For a command line Python script, making sure Python itself uses UTF-8
for the C locale is sufficient, and setting LC_CTYPE so spawned
processes that aren't Python have a chance at doing the right thing
too is a reasonable upgrade. This is probably good enough to drop one
hack[1] rather than porting it to Python 3.

For hosted Python code this does nothing (apart from print to stderr),
so mod_wsgi for instance is still going to need the same kind dance to
get users to set LANG as configuration themselves. Ideally this PEP
would have a C api or something so I could file bugs to make it just
do the right thing.

A few notes on specifics, I'll try not to stray too much into choices
already made.

The PEP doesn't persuade me that Py_Initialize actually is too late to
switch *specifically* from ascii to utf-8. Any preceeding operations
that operate on unicode would have been a safe subset. There might be
issues with other internals, or surrogateescape, or it's just a pain?

I don't like the side effect of changing the standard stream error
handler to surrogateescape if LANG=C.UTF-8 is actually set. Obviously
bad data vs exception is a trade off anyway, but means to get a Python
script that will always output valid data or exit, you have to set an
arbitrary language like en_US. Yes, that's true of the change as
implemented in 3.5 anyway.

Not setting LANG and only setting LC_CTYPE seems fine. Obviously,
things can go wrong based on odd behaviours of spawned processes, but
it works for the normal idioms.

I'm not sold on adding the PYTHONCOERCECLOCALE runtime configuration.
All it really does is turn off stderr kipple if you must use the C
locale for other reasons? Anyone with the ability to set that variable
could just set LANG instead. I was going to suggest just documenting
LC_ALL=C as the override instead of adding a python specific variable,
but note looking around that Debian discourage that[3].

That's all, though I will also grumble a bit about how long the PEP is.

Martin


[1] Override Py_FileSystemDefaultEncoding to utf-8 from ascii for the bzr script

[2] WSGIDaemonProcess lang and locale options

[3] "Using LC_ALL is strongly discouraged as it overrides everything"

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 538 (review round 2): Coercing the legacy C locale to a UTF-8 based locale

2017-06-11 Thread Nick Coghlan 
On 12 June 2017 at 10:05, Martin (gzlist) via Python-Dev
 wrote:
> I don't like the side effect of changing the standard stream error
> handler to surrogateescape if LANG=C.UTF-8 is actually set. Obviously
> bad data vs exception is a trade off anyway, but means to get a Python
> script that will always output valid data or exit, you have to set an
> arbitrary language like en_US. Yes, that's true of the change as
> implemented in 3.5 anyway.

`PYTHONIOENCODING=:strict` remains the preferred way of forcing strict
encoding checks on the standard streams, regardless of locale.

> I'm not sold on adding the PYTHONCOERCECLOCALE runtime configuration.
> All it really does is turn off stderr kipple if you must use the C
> locale for other reasons? Anyone with the ability to set that variable
> could just set LANG instead. I was going to suggest just documenting
> LC_ALL=C as the override instead of adding a python specific variable,
> but note looking around that Debian discourage that[3].

In addition to providing a reliable escape hatch with no other
potentially unwanted side effects (for when folks actually want the
current behaviour), the entry for the off switch in the CLI usage docs
also provides us with a convenient place to document the *default*
behaviour.

> That's all, though I will also grumble a bit about how long the PEP is.

The ASCII-to-Unicode migration has been in progress for almost as long
as Python has been around, and ASCII has been the default encoding in
C for almost twice as long as that, so it takes a bit of text to
explain why *now* is a good time to break with 50+ years of precedent
:)

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com