Re: [Python-Dev] Making stdlib modules optional for distributions (Was: Breaking up the stdlib (Was: release cadence))

2016-07-07 Thread Eric V. Smith 

On 7/6/2016 10:44 PM, Nick Coghlan wrote:

The point of overlap I see is that if the stdlib starts putting some
selected modules into site-packages (so "pip install --upgrade
" works without any changes to pip or equivalent tools),
then that also solves the "How to explicitly declare dependencies on
particular pieces of the standard library" problem: you use the same
mechanisms we already use to declare dependencies on 3rd party
packages distributed via a packaging server.


One thing to keep in mind if we do this is how it interacts with the -S 
command line option to not include site-packages in sys.path. I 
currently use -S to basically mean "give my python as it was 
distributed, and don't include anything that was subsequently added by 
adding other RPM's (or package manager of your choice)". I realize 
that's a rough description, and possibly an abuse of -S. If using -S 
were to start excluding parts of the stdlib, that would be a problem for me.


Eric.

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Making stdlib modules optional for distributions (Was: Breaking up the stdlib (Was: release cadence))

2016-07-07 Thread Barry Warsaw 
On Jul 07, 2016, at 08:12 AM, Eric V. Smith wrote:

>One thing to keep in mind if we do this is how it interacts with the -S
>command line option to not include site-packages in sys.path. I currently use
>-S to basically mean "give my python as it was distributed, and don't include
>anything that was subsequently added by adding other RPM's (or package
>manager of your choice)". I realize that's a rough description, and possibly
>an abuse of -S. If using -S were to start excluding parts of the stdlib, that
>would be a problem for me.

It's an important consideration, and leads to another discussion that's
recurred over the years.  Operating systems often want an "isolated" Python,
similar to what's given by -I, which cannot be altered by subsequent
installs.  It's one of the things that lead to the Debian ecosystem using
dist-packages for PyPI installed packages.  Without isolation, it's just too
easy for some random PyPI thing to break your system, and yes, that has really
happened in the past.

So if we go down the path of moving more of the stdlib to site-packages, we
also need to get serious about a system-specific isolated Python.

Cheers,
-Barry
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Making stdlib modules optional for distributions (Was: Breaking up the stdlib (Was: release cadence))

2016-07-07 Thread Daniel Holth 
Yes, not too long ago I installed "every" Python package on Ubuntu, and
Python basically would not start. Perhaps some plugin system was trying to
import everything and caused a segfault in GTK. The "short sys.path" model
of everything installed is importable has its limits.

On Thu, Jul 7, 2016 at 9:24 AM Barry Warsaw  wrote:

> On Jul 07, 2016, at 08:12 AM, Eric V. Smith wrote:
>
> >One thing to keep in mind if we do this is how it interacts with the -S
> >command line option to not include site-packages in sys.path. I currently
> use
> >-S to basically mean "give my python as it was distributed, and don't
> include
> >anything that was subsequently added by adding other RPM's (or package
> >manager of your choice)". I realize that's a rough description, and
> possibly
> >an abuse of -S. If using -S were to start excluding parts of the stdlib,
> that
> >would be a problem for me.
>
> It's an important consideration, and leads to another discussion that's
> recurred over the years.  Operating systems often want an "isolated"
> Python,
> similar to what's given by -I, which cannot be altered by subsequent
> installs.  It's one of the things that lead to the Debian ecosystem using
> dist-packages for PyPI installed packages.  Without isolation, it's just
> too
> easy for some random PyPI thing to break your system, and yes, that has
> really
> happened in the past.
>
> So if we go down the path of moving more of the stdlib to site-packages, we
> also need to get serious about a system-specific isolated Python.
>
> Cheers,
> -Barry
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/dholth%40gmail.com
>
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python parser performance optimizations

2016-07-07 Thread Artyom Skrobov 
Hello,

This is a monthly ping to get a review on http://bugs.python.org/issue26415 -- 
"Excessive peak memory consumption by the Python parser".
The first patch of the series (an NFC refactoring) was successfully committed 
earlier in June, so the next step is to get the second patch, "the payload", 
reviewed and committed.

To address the concerns raised by the commenters back in May: the patch doesn't 
lead to negative memory consumption, of course. The base for calculating 
percentages is the smaller number of the two; this is the same style of 
reporting that perf.py uses. In other words, "200% less memory usage" is a 
threefold shrink.

The absolute values, and the way they were produced, are all reported under the 
ticket.


From: Artyom Skrobov
Sent: 26 May 2016 11:19
To: '[email protected]'
Subject: Python parser performance optimizations

Hello,

Back in March, I've posted a patch at http://bugs.python.org/issue26526 -- "In 
parsermodule.c, replace over 2KLOC of hand-crafted validation code, with a DFA".

The motivation for this patch was to enable a memory footprint optimization, 
discussed at http://bugs.python.org/issue26415
My proposed optimization reduces the memory footprint by up to 30% on the 
standard benchmarks, and by 200% on a degenerate case which sparked the 
discussion.
The run time stays unaffected by this optimization.

Python Developer's Guide says: "If you don't get a response within a few days 
after pinging the issue, then you can try emailing 
[email protected] asking for someone to 
review your patch."

So, here I am.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Debugging Python scripts with GDB on OSX

2016-07-07 Thread Alexandru Croitor 
Hello,

I'm interested to find out if debugging Python scripts with GDB is supported on 
OSX at all?

I'm referring to the functionality described on 
https://wiki.python.org/moin/DebuggingWithGdb and on 
http://fedoraproject.org/wiki/Features/EasierPythonDebugging.

I've tried so far various combinations of pre-compiled GDB from the homebrew 
package manager, locally-compiled GDB from homebrew, as well as locally 
compiled GDB from MacPorts, together with a pre-compiled Python 2.7, 
homebrew-compiled 2.7, and custom compiled Python 2.7 from the official source 
tarball.

My results so far were not successful. The legacy GDB commands to show a python 
stack trace or the local variables - do not work. And the new GDB commands 
(referenced on the Fedora project page) are not present at all in any of the 
GDB versions.

I've checked the python CI build bot tests, and it seems the new GDB commands 
are only successfully tested on Linux machines, and are skipped on FreeBSD, OS 
X, and Solaris machines.

Are the new python <-> GDB commands specific to Linux?
Are there any considerations to take in regards to debug symbols for Python / 
GDB on OSX?

Has anyone attempted what I'm trying to do?

I would be grateful for any advice.

And I apologize if my choice of the mailing lists is not the best.

Regards, Alex.


___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Making stdlib modules optional for distributions (Was: Breaking up the stdlib (Was: release cadence))

2016-07-07 Thread Steve Dower 

On 07Jul2016 0624, Barry Warsaw wrote:

On Jul 07, 2016, at 08:12 AM, Eric V. Smith wrote:


One thing to keep in mind if we do this is how it interacts with the -S
command line option to not include site-packages in sys.path. I currently use
-S to basically mean "give my python as it was distributed, and don't include
anything that was subsequently added by adding other RPM's (or package
manager of your choice)". I realize that's a rough description, and possibly
an abuse of -S. If using -S were to start excluding parts of the stdlib, that
would be a problem for me.


It's an important consideration, and leads to another discussion that's
recurred over the years.  Operating systems often want an "isolated" Python,
similar to what's given by -I, which cannot be altered by subsequent
installs.  It's one of the things that lead to the Debian ecosystem using
dist-packages for PyPI installed packages.  Without isolation, it's just too
easy for some random PyPI thing to break your system, and yes, that has really
happened in the past.

So if we go down the path of moving more of the stdlib to site-packages, we
also need to get serious about a system-specific isolated Python.


I've done just enough research to basically decide that putting any of 
the stdlib in site-packages is infeasible (it'll break virtualenv/venv 
as well), so don't worry about that.


A "dist-packages" equivalent is a possibility, and it may even be 
possible to manage these packages directly in Lib/, which would result 
in basically no visible impact for anyone who doesn't care to update 
individual parts.


Cheers,
Steve


Cheers,
-Barry


___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Debugging Python scripts with GDB on OSX

2016-07-07 Thread R. David Murray 
On Wed, 06 Jul 2016 16:14:34 -, Alexandru Croitor  
wrote:
> I'm interested to find out if debugging Python scripts with GDB is supported 
> on OSX at all?
> 
> I'm referring to the functionality described on 
> https://wiki.python.org/moin/DebuggingWithGdb and on 
> http://fedoraproject.org/wiki/Features/EasierPythonDebugging.
> 
> I've tried so far various combinations of pre-compiled GDB from the homebrew 
> package manager, locally-compiled GDB from homebrew, as well as locally 
> compiled GDB from MacPorts, together with a pre-compiled Python 2.7, 
> homebrew-compiled 2.7, and custom compiled Python 2.7 from the official 
> source tarball.
> 
> My results so far were not successful. The legacy GDB commands to show a 
> python stack trace or the local variables - do not work. And the new GDB 
> commands (referenced on the Fedora project page) are not present at all in 
> any of the GDB versions.
> 
> I've checked the python CI build bot tests, and it seems the new GDB commands 
> are only successfully tested on Linux machines, and are skipped on FreeBSD, 
> OS X, and Solaris machines.
> 
> Are the new python <-> GDB commands specific to Linux?
> Are there any considerations to take in regards to debug symbols for Python / 
> GDB on OSX?
> 
> Has anyone attempted what I'm trying to do?
> 
> I would be grateful for any advice.
> 
> And I apologize if my choice of the mailing lists is not the best.

I tried to do this a few weeks ago myself, with similar negative
results.  The only thing I tried that you don't mention (I didn't
try everything you did) is a compile from raw gdb source...and that
didn't support OSX format core dumps.  So I gave up.

--David
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] __qualname__ exposed as a local variable: standard?

2016-07-07 Thread Carlos Pita 
Hi all,

I noticed __qualname__ is exposed by locals() while defining a class. This
is handy but I'm not sure about its status: is it standard or just an
artifact of the current implementation? (btw, the pycodestyle linter
-former pep8- rejects its usage). I was unable to find any reference to
this behavior in PEP 3155 nor in the language reference.

Thank you in advance
--
Carlos
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com