Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3
2012/2/10 Eli Bendersky > > > > Thanks for the input, Florent. So, to paraphrase, there already are > code changes in the stdlib version of ET/cET which are not upstream. > You made it explicit about the tests, so the question is only left for > the modules themselves. Is that right? > The port of ElementTree to Python 3000 was done in the standard library only. The work was done back in 2006, 2007 and 2008. There was never a public version of ElementTree for Python 3 outside of the standard library. It is already a significant change from the upstream branch (many changes in the C extension code). Then when I enforced the same test suite for both implementation, I have fixed many things in the C extension module too. To my knowledge, these fixes were not included upstream. Since two years, there was regular maintenance of the package in the standard library, but none of the patch were integrated upstream. I hope it answers the question, -- Florent Xicluna ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a new "locale" codec?
Victor Stinner writes: > > If this is needed, it should be spelled "os.getlocaleencoding()" (or > > "sys.getlocaleencoding()"?) > > There is already a locale.getpreferredencoding(False) function which > give your the current locale encoding. The problem is that the current > locale encoding may change and so you have to get the new value each > time than you would like to encode or decode data. How can that happen if the programmer (or a module she has imported) isn't messing with the locale? If the programmer is messing with the locale, really they need to be careful. A magic codec whose encoding changes *within* a process is an accident waiting to happen. Do you have a real use case for the "'locale' codec's encoding changes with the locale within process" feature? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3
On Fri, Feb 10, 2012 at 10:32, Florent wrote: > 2012/2/10 Eli Bendersky >> > >> >> Thanks for the input, Florent. So, to paraphrase, there already are >> code changes in the stdlib version of ET/cET which are not upstream. >> You made it explicit about the tests, so the question is only left for >> the modules themselves. Is that right? >> > > The port of ElementTree to Python 3000 was done in the standard > library only. The work was done back in 2006, 2007 and 2008. > There was never a public version of ElementTree for Python 3 outside > of the standard library. > It is already a significant change from the upstream branch (many > changes in the C extension code). > > Then when I enforced the same test suite for both implementation, I > have fixed many things in the C extension module too. To my knowledge, > these fixes were not included upstream. > > Since two years, there was regular maintenance of the package in the > standard library, but none of the patch were integrated upstream. > Folks, with this in mind, can we just acknowledge that the stdlib ElementTree is de-facto forked from Fredrik Lundh's official releases and get on with our lives? Note the code review discussion here - http://codereview.appspot.com/207048/show - where Fredrik Lundh more or less acknowledges this fact and shows no real objections to it. By "get on with our lives" I mean keep fixing problems in ElementTree inside stdlib, as well as work on exposing the C implementation behind the ElementTree API by default, falling back on the Python API (and being true to PEP 399). Eli ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3
> That makes me consider it the reality that "today, ET is only being > maintained in the stdlib". I think different people will have different perceptions of reality here. In my interaction with Fredrik Lundh, I got the impression that he might consider code still maintained even if he didn't touch it for five and more years - as he might get back some time and work on it, and as it may not have significant bugs that need fixing. If someone steps into charge and actually takes over ElementTree maintainance, that would be fine with me (as long as I have some trust that he'll continue to work on it for the next five years). We might have to "write off" contributions from Fredrik Lundh to Python because of that. Notice that the last time something like this came up (bsddb), it actually resulted in a removal of the respective package from the standard library. > Given that it was two months ago that I started the "Fixing the XML > batteries" thread (and years since I brought up the topic for the first > time), it seems to be hard enough already to get anyone on python-dev > actually do something for Python's XML support, instead of just actively > discouraging those who invest time and work into it. It depends on the nature of the changes you want to see done. Just bashing some piece of code is not something that I personally consider a worthwhile thing, so I'll likely continue to discourage changes in a direction that demeans some XML library in favor of some other. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PyPy 1.8 released
PyPy 1.8 - business as usual We're pleased to announce the 1.8 release of PyPy. As habitual this release brings a lot of bugfixes, together with performance and memory improvements over the 1.7 release. The main highlight of the release is the introduction of `list strategies`_ which makes homogenous lists more efficient both in terms of performance and memory. This release also upgrades us from Python 2.7.1 compatibility to 2.7.2. Otherwise it's "business as usual" in the sense that performance improved roughly 10% on average since the previous release. you can download the PyPy 1.8 release here: http://pypy.org/download.html .. _`list strategies`: http://morepypy.blogspot.com/2011/10/more-compact-lists-with-list-strategies.html What is PyPy? = PyPy is a very compliant Python interpreter, almost a drop-in replacement for CPython 2.7. It's fast (`pypy 1.8 and cpython 2.7.1`_ performance comparison) due to its integrated tracing JIT compiler. This release supports x86 machines running Linux 32/64, Mac OS X 32/64 or Windows 32. Windows 64 work has been stalled, we would welcome a volunteer to handle that. .. _`pypy 1.8 and cpython 2.7.1`: http://speed.pypy.org Highlights == * List strategies. Now lists that contain only ints or only floats should be as efficient as storing them in a binary-packed array. It also improves the JIT performance in places that use such lists. There are also special strategies for unicode and string lists. * As usual, numerous performance improvements. There are many examples of python constructs that now should be faster; too many to list them. * Bugfixes and compatibility fixes with CPython. * Windows fixes. * NumPy effort progress; for the exact list of things that have been done, consult the `numpy status page`_. A tentative list of things that has been done: * multi dimensional arrays * various sizes of dtypes * a lot of ufuncs * a lot of other minor changes Right now the `numpy` module is available under both `numpy` and `numpypy` names. However, because it's incomplete, you have to `import numpypy` first before doing any imports from `numpy`. * New JIT hooks that allow you to hook into the JIT process from your python program. There is a `brief overview`_ of what they offer. * Standard library upgrade from 2.7.1 to 2.7.2. Ongoing work As usual, there is quite a bit of ongoing work that either didn't make it to the release or is not ready yet. Highlights include: * Non-x86 backends for the JIT: ARMv7 (almost ready) and PPC64 (in progress) * Specialized type instances - allocate instances as efficient as C structs, including type specialization * More numpy work * Since the last release there was a significant breakthrough in PyPy's fundraising. We now have enough funds to work on first stages of `numpypy`_ and `py3k`_. We would like to thank again to everyone who donated. * It's also probably worth noting, we're considering donations for the Software Transactional Memory project. You can read more about `our plans`_ Cheers, The PyPy Team .. _`brief overview`: http://doc.pypy.org/en/latest/jit-hooks.html .. _`numpy status page`: http://buildbot.pypy.org/numpy-status/latest.html .. _`numpy status update blog report`: http://morepypy.blogspot.com/2012/01/numpypy-status-update.html .. _`numpypy`: http://pypy.org/numpydonate.html .. _`py3k`: http://pypy.org/py3donate.html .. _`our plans`: http://morepypy.blogspot.com/2012/01/transactional-memory-ii.html ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3
> IMHO it's no longer a question of "wanting" to take ownership. > According to Florent, this has already happened to some extent. "Ownership to some extent" is not a useful concept. Either you have ownership, or you don't. > I don't mind sending Fredrik an email as you detailed. Any suggested > things to include in it? I'd ask Fredrik if he wants to yield ownership, to some (specific) other person. What really worries me is the question who that other person is. There is a difference between fixing some issues, and actively taking over ownership, with all its consequences (long-term commitment, willingness to defend difficult decisions even if you are constantly being insulted for that decision, and so on). *Not* having an owner just means that it will be as unmaintained in the future as it appears to be now. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3
Eli Bendersky, 10.02.2012 10:06: > On Fri, Feb 10, 2012 at 10:32, Florent wrote: >> 2012/2/10 Eli Bendersky >>> >>> Thanks for the input, Florent. So, to paraphrase, there already are >>> code changes in the stdlib version of ET/cET which are not upstream. >>> You made it explicit about the tests, so the question is only left for >>> the modules themselves. Is that right? >>> >> >> The port of ElementTree to Python 3000 was done in the standard >> library only. The work was done back in 2006, 2007 and 2008. >> There was never a public version of ElementTree for Python 3 outside >> of the standard library. >> It is already a significant change from the upstream branch (many >> changes in the C extension code). >> >> Then when I enforced the same test suite for both implementation, I >> have fixed many things in the C extension module too. To my knowledge, >> these fixes were not included upstream. >> >> Since two years, there was regular maintenance of the package in the >> standard library, but none of the patch were integrated upstream. > > Folks, with this in mind, can we just acknowledge that the stdlib > ElementTree is de-facto forked from Fredrik Lundh's official releases > and get on with our lives? Note the code review discussion here - > http://codereview.appspot.com/207048/show - where Fredrik Lundh more > or less acknowledges this fact and shows no real objections to it. > > By "get on with our lives" I mean keep fixing problems in ElementTree > inside stdlib, as well as work on exposing the C implementation behind > the ElementTree API by default, falling back on the Python API (and > being true to PEP 399). +1 None of this would make the situation any worse than it currently is, but provide serious improvements to the user experience. Stefan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a new "locale" codec?
> As And pointed out, this is already the behaviour of the "mbcs" codec > under Windows. "locale" would be the moral (*) equivalent of that under > Unix. Indeed, and that precedent should be enough reason *not* to include a "locale" encoding. The "mbcs" encoding has caused much user confusion over the years, and it is less useful than people typically think. For example, for some time, people thought that names in zip files ought to be encoded in "mbcs", only to find out that this is incorrect years later. With a "locale" encoding, the risk for confusion and untestable code is too high (just consider the ongoing saga of the Turkish dotless i (ı)). Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3
On Fri, Feb 10, 2012 at 11:43, "Martin v. Löwis" wrote: >> IMHO it's no longer a question of "wanting" to take ownership. >> According to Florent, this has already happened to some extent. > > "Ownership to some extent" is not a useful concept. Either you have > ownership, or you don't. > >> I don't mind sending Fredrik an email as you detailed. Any suggested >> things to include in it? > > I'd ask Fredrik if he wants to yield ownership, to some (specific) other > person. > > What really worries me is the question who that other person is. There > is a difference between fixing some issues, and actively taking over > ownership, with all its consequences (long-term commitment, willingness > to defend difficult decisions even if you are constantly being insulted > for that decision, and so on). > > *Not* having an owner just means that it will be as unmaintained in > the future as it appears to be now. How does this differ from any other module in stdlib that may not have a single designated owner, but which at the same time *is* being maintained by the core developers as a group? ISTM that requiring a five-year commitment is just going to scare any contributors away - is that what we want? What worries me most is that there seems to be a flow towards status quo on such things because status quo is the easiest to do. But in some cases, status quo is bad. Here we have a quite popular package in stdlib whose maintainer stopped maintaining it about two years ago. Another person stepped up and did some good work to bring the package up to date, fix bugs, and improve the test suite. What happens now? Do we give up on touching it until Fredrik Lundh decides on a come-back or some person who is willing to commit 5 years is found? Or do we just *keep* maintaining it in the stdlib as we do with other modules, fixing bugs, tests, documentation and so on? Eli ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] maintenance of the ElementTree / cElementTree packages in the Python standard library
Hello Fredrik, Recently a discussion came up on the python-dev mailing list regarding continued maintenance of the ElementTree & cElementTree packages which are part of the standard library, and which were originally contributed by you. There currently exists an unclear situation with respect to the active maintainer(s) of these packages. On one hand, PEP 360 states that the packages are officially maintained in your repositories and all problems should be assigned to you and fixed upstream. On the other hand, it appears that there has already been made a considerable amount of work (e.g. http://codereview.appspot.com/207048/show) solely within the Python repositories, as well as the port to Python 3 which now lives in all 3.x branches. In other words, de-facto the package has been forked in the Python repository. Note that no changes (AFAIU) have been made to the ElementTree *API*, only to the implementations living in the stdlib. I'd like to understand your point of view on this topic. There are currently 23 open issues on the package(s) in the Python tracker, and some additional plans are being made (such as 'import ElementTree' importing the C implementation by default, falling back on the Python implementation if that's unavailable). Is that alright with you that all such new fixes and developments are being made by Python code developers in the Python repositories directly, without waiting for your approval to submit them upstream? Thanks in advance, Eli ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3
> How does this differ from any other module in stdlib that may not have > a single designated owner, but which at the same time *is* being > maintained by the core developers as a group? ISTM that requiring a > five-year commitment is just going to scare any contributors away - is > that what we want? I'm not talking about contributors, I'm talking about a maintainer. When we have a maintainer, that can actually attract contributors. Recognizing that something is a long-term commitment is scary, yes. However, a number of contributors has accepted such commitments, e.g. the release managers. Asking that for a subpackage is not asked too much, IMO. Compare this to distutils: if we've had a commitment of the original author to maintain that for a long period, the setuptools, distribute, distutils2, and packaging forks may not have been necessary. In absence of a maintainer, nobody is able to make difficult decisions. > What happens now? Do we give up on touching it until Fredrik Lundh > decides on a come-back or some person who is willing to commit 5 years > is found? Or do we just *keep* maintaining it in the stdlib as we do > with other modules, fixing bugs, tests, documentation and so on? If we really can't find somebody dedicated to that code base enough, we should consider removing it from the standard library. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3
"Martin v. Löwis", 10.02.2012 10:37: >> Given that it was two months ago that I started the "Fixing the XML >> batteries" thread (and years since I brought up the topic for the first >> time), it seems to be hard enough already to get anyone on python-dev >> actually do something for Python's XML support, instead of just actively >> discouraging those who invest time and work into it. > > It depends on the nature of the changes you want to see done. Just > bashing some piece of code is not something that I personally consider > a worthwhile thing, so I'll likely continue to discourage changes in > a direction that demeans some XML library in favor of some other. This is getting off-topic for this thread, but anyway. What I meant with my paragraph above was that none of the topics I brought up has received any action on the side of those with commit rights yet, regardless of how obvious they were and how much or little dispute there was about them. IMHO, all of this boils down to whether or not we should make it easier for users to efficiently use the stdlib. Backing ElementTree by the accelerator module helps here, and fixing the docs to point (new) users to ElementTree instead of MiniDOM helps as well. I can happily accept that you have a different opinion on the latter topic than I do. What I cannot accept is that, as we speak, this leads to users getting drawn into using the wrong tool for their job, into wasting their time (both for development and runtime) and potentially into getting drawn away from the (IMHO) perfect language for XML processing. I don't think bashing is the right word here. Everyone who, knowing the alternatives, decides to use MiniDOM is welcome to do so. I'm just stating, both from my personal experience and from discussions on c.l.py, that the current documentation makes it easier for new users to take the wrong decision for them than to make this decision in an informed way. MiniDOM *may* be the right thing further down along the way in some cases. It's almost *never* the right thing to start with, simply because if you do, it inherently takes way too much time until you reach the point where the evidence becomes obvious that it actually was the wrong decision. The documentation should allow innocent users to see this risk clearly before they start wasting their time. So, getting back to the topic again, is there any reason why you would oppose backing the ElementTree module in the stdlib by cElementTree's accelerator module? Or can we just consider this part of the discussion settled and start getting work done? Stefan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3
>> What happens now? Do we give up on touching it until Fredrik Lundh >> decides on a come-back or some person who is willing to commit 5 years >> is found? Or do we just *keep* maintaining it in the stdlib as we do >> with other modules, fixing bugs, tests, documentation and so on? > > If we really can't find somebody dedicated to that code base enough, > we should consider removing it from the standard library. Does this imply that each and every package in the stdlib currently has a dedicated maintainer who promised to be dedicated to it? Or otherwise, should those packages that *don't* have a maintainer be removed from the standard library? Isn't that a bit harsh? ElementTree is an overall functional library and AFAIK the preferred stdlib tool for processing XML for many developers. It currently needs some attention to fix a few issues, expose the fast C implementation by default when ElementTree is imported, and improve the documentation. At this point, I'm interested enough to work on these - given that the political issue with Fredrik Lundh is resolved. However, I can't *honestly* say I promise to maintain the package until 2017. So, what's next? Eli ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3
"Martin v. Löwis", 10.02.2012 11:32: >> What happens now? Do we give up on touching it until Fredrik Lundh >> decides on a come-back or some person who is willing to commit 5 years >> is found? Or do we just *keep* maintaining it in the stdlib as we do >> with other modules, fixing bugs, tests, documentation and so on? > > If we really can't find somebody dedicated to that code base enough, > we should consider removing it from the standard library. Well, that's totally not the current situation, though. There has been a large amount of maintenance going into the ElementTree modules already, so there is evidently a substantial interest in a) having them in the stdlib and b) keeping them working well. The current decisions could easily be taken by the interested parties, of which there seem to be enough involved in the relevant python-dev threads so far. Note that even decisions taken by a maintainer are not guaranteed to pass easily and without opposition. On a related note, it may be worth mentioning that it's generally known for several years now that the MiniDOM library has very serious performance problems, and there doesn't seem to be any maintainer around who has made a visible effort to solve them. Maybe we should remove MiniDOM from the stdlib, because no-one seems to be dedicated to that code base enough to fix it. Stefan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Fwd: maintenance of the ElementTree / cElementTree packages in the Python standard library
-- Forwarded message -- From: Fredrik Lundh Date: Fri, Feb 10, 2012 at 13:16 Subject: Re: maintenance of the ElementTree / cElementTree packages in the Python standard library To: Eli Bendersky Hi Eli, thanks for reaching out. I'll get back to you with a more "formal" reply later, but yeah, that sounds like a plan -- I have very limited time for core Python work these days anyway (as you guys have probably noticed :-). But feel free to loop me in on suggested API changes going forward (and I'll dig up my notes on additions that would be nice to have if someone wants to work on that). 2012/2/10 Eli Bendersky : > Hello Fredrik, > > Recently a discussion came up on the python-dev mailing list regarding > continued maintenance of the ElementTree & cElementTree packages which > are part of the standard library, and which were originally > contributed by you. > > There currently exists an unclear situation with respect to the active > maintainer(s) of these packages. On one hand, PEP 360 states that the > packages are officially maintained in your repositories and all > problems should be assigned to you and fixed upstream. On the other > hand, it appears that there has already been made a considerable > amount of work (e.g. http://codereview.appspot.com/207048/show) solely > within the Python repositories, as well as the port to Python 3 which > now lives in all 3.x branches. In other words, de-facto the package > has been forked in the Python repository. Note that no changes (AFAIU) > have been made to the ElementTree *API*, only to the implementations > living in the stdlib. > > I'd like to understand your point of view on this topic. There are > currently 23 open issues on the package(s) in the Python tracker, and > some additional plans are being made (such as 'import ElementTree' > importing the C implementation by default, falling back on the Python > implementation if that's unavailable). Is that alright with you that > all such new fixes and developments are being made by Python code > developers in the Python repositories directly, without waiting for > your approval to submit them upstream? > > Thanks in advance, > Eli ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] peps: Update with bugfix releases.
In article , Ned Deily wrote: > However, this may all be a moot point now as I've subsequently proposed > a patch to Distutils to smooth over the problem by checking for the case > of gcc-4.2 being required but not available and, if so, automatically > substituting clang instead. (http://bugs.python.org/issue13590) This > trades off a certain risk of using clang for extension modules against > the 100% certainty of users being unable to build extension modules. And I've now committed the patch for 2.7.x and 3.2.x so I no longer consider this a release blocking issue for 2.7.3 and 3.2.3. -- Ned Deily, [email protected] ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3
On Fri, Feb 10, 2012 at 7:37 PM, "Martin v. Löwis" wrote: > Notice that the last time something like this came up (bsddb), it > actually resulted in a removal of the respective package from the > standard library. bsddb was a *very* different case - it was actively causing buildbot stability problems and various reports on the tracker due to changes in the external Berkeley DB API. Once we had sqlite3 in the standard lib as an alternate DB-API backend, it was hard to justify the ongoing maintenance hassles *despite* Jesus Cea stepping up as the maintainer (and he still maintains the pybsddb version - that was actually a big factor in *letting* us drop it, since we could just direct current users towards the PyPI version). Most orphan modules in the stdlib aren't like that - yes, their APIs stagnate (because nobody feels they have the authority and/or expertise to make potentially controversial decisions), but for many of them, that's not a particularly bad thing. For others, the world has moved on around them and they becomes traps for the unwary, but still, taking the modules out is unwarranted, since we'd be breaking code without giving affected users a good alternative (for orphan modules, nobody is likely to take the time to maintain them on PyPI if they weren't willing to do so in the stdlib - this actually stands in stark *contrast* to the bsddb case, which was decidedly *not* an orphan module when it was removed). Regards, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3
2012/2/10 Nick Coghlan : > > Most orphan modules in the stdlib aren't like that - yes, their APIs > stagnate (because nobody feels they have the authority and/or > expertise to make potentially controversial decisions), but for many > of them, that's not a particularly bad thing. You're right, and sometimes a contributor steps in and propose a PEP to move things forward for a so-called orphan module. If I'm not wrong, it was the case for StringIO, pickle, distutils, wsgiref and optparse even if each of these packages has its own story. -- Florent Xicluna ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 411: Provisional packages in the Python standard library
Hi all, Following the intensive and fruitful discussion of the (now rejected) PEP 408 (http://mail.python.org/pipermail/python-dev/2012-January/115850.html), we've drafted PEP 411 to summarize the conclusions with regards to the process of marking packages provisional. Note that this is an informational PEP, and that for the sake of completeness it duplicates some of the contents of PEP 408. It is pasted below, as well as online at http://www.python.org/dev/peps/pep-0411/. Comments are welcome. Eli PEP: 411 Title: Provisional packages in the Python standard library Version: $Revision$ Last-Modified: $Date$ Author: Nick Coghlan , Eli Bendersky Status: Draft Type: Informational Content-Type: text/x-rst Created: 2012-02-10 Python-Version: 3.3 Post-History: 2012-02-10 Abstract The process of including a new package into the Python standard library is hindered by the API lock-in and promise of backward compatibility implied by a package being formally part of Python. This PEP describes a methodology for marking a standard library package "provisional" for the period of a single minor release. A provisional package may have its API modified prior to "graduating" into a "stable" state. On one hand, this state provides the package with the benefits of being formally part of the Python distribution. On the other hand, the core development team explicitly states that no promises are made with regards to the the stability of the package's API, which may change for the next release. While it is considered an unlikely outcome, such packages may even be removed from the standard library without a deprecation period if the concerns regarding their API or maintenante prove well-founded. Proposal - a documented provisional state = Whenever the Python core development team decides that a new package should be included into the standard library, but isn't entirely sure about whether the package's API is optimal, the package can be included and marked as "provisional". In the next minor release, the package may either be "graduated" into a normal "stable" state in the standard library, or be rejected and removed entirely from the Python source tree. If the package ends up graduating into the stable state after being provisional for a minor release, its API may be changed according to accumulated feedback. The core development team explicitly makes no guarantees about API stability and backward compatibility of provisional packages. Marking a package provisional - A package will be marked provisional by including the following paragraph as a note at the top of its documentation page: The package has been included in the standard library on a provisional basis. While major changes are not anticipated, as long as this notice remains in place, backwards incompatible changes are permitted if deemed necessary by the standard library developers. Such changes will not be made gratuitously - they will occur only if serious API flaws are uncovered that were missed prior to inclusion of the package. Moving a package from the provisional to the stable state simply implies removing this note from its documentation page. Which packages should go through the provisional state -- We expect most packages proposed for addition into the Python standard library to go through a minor release in the provisional state. There may, however, be some exceptions, such as packages that use a pre-defined API (for example ``lzma``, which generally follows the API of the existing ``bz2`` package), or packages with an API that has wide acceptance in the Python development community. In any case, packages that are proposed to be added to the standard library, whether via the provisional state or directly, must fulfill the acceptance conditions set by PEP 2. Criteria for "graduation" - In principle, most provisional packages should eventually graduate to the stable standard library. Some reasons for not graduating are: * The package may prove to be unstable or fragile, without sufficient developer support to maintain it. * A much better alternative package may be found during the preview release. Essentially, the decision will be made by the core developers on a per-case basis. The point to emphasize here is that a packages's inclusion in the standard library as "provisional" in some release does not guarantee it will continue being part of Python in the next release. Rationale = Benefits for the core development team -- Currently, the core developers are really reluctant to add new interfaces to the standard library. This is because as soon as they're published in a release, API design mistakes get locked in due to backward compatibility c
Re: [Python-Dev] Fwd: maintenance of the ElementTree / cElementTree packages in the Python standard library
On Fri, Feb 10, 2012 at 9:26 PM, Eli Bendersky wrote: > -- Forwarded message -- > From: Fredrik Lundh > Date: Fri, Feb 10, 2012 at 13:16 > Subject: Re: maintenance of the ElementTree / cElementTree packages in > the Python standard library > To: Eli Bendersky > > > Hi Eli, thanks for reaching out. I'll get back to you with a more > "formal" reply later, but yeah, that sounds like a plan -- I have very > limited time for core Python work these days anyway (as you guys have > probably noticed :-). I've updated PEP 360 accordingly (including a link back to the archived version of Fredrik's reply). Since ElementTree was the last Python module referenced from that PEP that hadn't been converted to python-dev maintenance, I flagged the PEP so it now appears in the Historical PEPs section rather than near the top of the PEP index. Technically the reference from there to the Expat XML parser being externally maintained is still valid, but the same could be said of various 3rd party libraries we ship with the Windows binaries. I also updated the headers on several old PEPs (mostly ones related specifically to the 3.0 process and the migration to Hg) to move them down into the Historical section, and fixed the PEP 0 generator so that Draft process PEPs (i.e. the PEP 407 proposal to change the release schedule) appear in the Open PEPs section along with all the other Draft PEPs. (At time of writing, the PEP pages hadn't regenerated to show the updated status of any of the PEPs I moved around, but I figure it will sort itself out eventually) Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3
> """A common pattern in Python 2.x is to have one version of a module > implemented in pure Python, with an optional accelerated version > implemented as a C extension; for example, pickle and cPickle. This > places the burden of importing the accelerated version and falling > back on the pure Python version on each user of these modules. In > Python 3.0, the accelerated versions are considered implementation > details of the pure Python versions. Users should always import the > standard version, which attempts to import the accelerated version and > falls back to the pure Python version. The pickle / cPickle pair > received this treatment. The profile module is on the list for 3.1. > The StringIO module has been turned into a class in the io module.""" > > Is there a good reason why xml.etree.ElementTree / > xml.etree.cElementTree did not "receive this treatment"? > Since there appeared to be an overall positive response for making this change in Python 3.3, and since there isn't longer any doubt about the ownership of the package *in Python's stdlib* (see http://mail.python.org/pipermail/python-dev/2012-February/116389.html), I've opened issue 13988 on the bug tracker to follow the implementation. Eli ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2012-02-03 - 2012-02-10) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open3246 ( -2) closed 22523 (+57) total 25769 (+55) Open issues with patches: 1394 Issues opened (36) == #13929: fnmatch to support escape characters http://bugs.python.org/issue13929 reopened by terry.reedy #13934: sqlite3 test typo http://bugs.python.org/issue13934 opened by poq #13938: 2to3 fails to convert types.StringTypes appropriately http://bugs.python.org/issue13938 opened by mhammond #13940: imaplib: Mailbox names are not quoted http://bugs.python.org/issue13940 opened by joebauer #13942: ssl.wrap_socket does not work on socket.socketpair()'s http://bugs.python.org/issue13942 opened by weary #13943: distutilsâ build_py fails when package string is unicode http://bugs.python.org/issue13943 opened by patrick.andrew #13946: readline completer could return an iterable http://bugs.python.org/issue13946 opened by nicolas_49 #13948: rm needless use of set function http://bugs.python.org/issue13948 opened by tshepang #13949: rm needless use of pass statement http://bugs.python.org/issue13949 opened by tshepang #13950: rm commented-out code http://bugs.python.org/issue13950 opened by tshepang #13951: Seg Fault in .so called by ctypes causes the interpreter to Se http://bugs.python.org/issue13951 opened by graemeglass #13952: mimetypes doesn't recognize .csv http://bugs.python.org/issue13952 opened by iwd32900 #13953: Get rid of doctests in packaging.tests.test_version http://bugs.python.org/issue13953 opened by tshepang #13954: Add regrtest option to record test results to a file http://bugs.python.org/issue13954 opened by brett.cannon #13959: Re-implement parts of imp in pure Python http://bugs.python.org/issue13959 opened by brett.cannon #13960: Handling of broken comments in HTMLParser http://bugs.python.org/issue13960 opened by ezio.melotti #13961: Have importlib use os.replace() http://bugs.python.org/issue13961 opened by brett.cannon #13962: multiple lib and include directories on Linux http://bugs.python.org/issue13962 opened by rpq #13963: dev guide has no mention of mechanics of patch review http://bugs.python.org/issue13963 opened by dmalcolm #13964: os.utimensat() and os.futimes() should accept Decimal, drop os http://bugs.python.org/issue13964 opened by haypo #13966: Add disable_interspersed_args() to argparse.ArgumentParser http://bugs.python.org/issue13966 opened by Laszlo.Attila.Toth #13967: also test for an empty pathname http://bugs.python.org/issue13967 opened by tshepang #13968: Support recursive globs http://bugs.python.org/issue13968 opened by ubershmekel #13969: path name must always be string (or None) http://bugs.python.org/issue13969 opened by tshepang #13970: frameobject should not have f_yieldfrom attribute http://bugs.python.org/issue13970 opened by Mark.Shannon #13972: set and frozenset constructors don't accept multiple iterables http://bugs.python.org/issue13972 opened by petri.lehtinen #13973: urllib.parse is imported twice in xmlrpc.client http://bugs.python.org/issue13973 opened by tshepang #13974: packaging: test for set_platform() http://bugs.python.org/issue13974 opened by tshepang #13977: importlib simplification http://bugs.python.org/issue13977 opened by Jim.Jewett #13978: OSError exception in multiprocessing module when using os.remo http://bugs.python.org/issue13978 opened by jjardon #13979: Automatic *libc.so loading behaviour http://bugs.python.org/issue13979 opened by dgoulet #13981: time.sleep() should use nanosleep() if available http://bugs.python.org/issue13981 opened by haypo #13985: Menu.tk_popup : menu doesn't disapear when main window is ico http://bugs.python.org/issue13985 opened by marc.dechico #13986: ValueError: cannot convert float NaN to integer http://bugs.python.org/issue13986 opened by shivam_python_issues #13987: Handling of broken markup in HTMLParser on 2.7 http://bugs.python.org/issue13987 opened by ezio.melotti #13988: Expose the C implementation of ElementTree by default when imp http://bugs.python.org/issue13988 opened by eli.bendersky Most recent 15 issues with no replies (15) == #13987: Handling of broken markup in HTMLParser on 2.7 http://bugs.python.org/issue13987 #13985: Menu.tk_popup : menu doesn't disapear when main window is ico http://bugs.python.org/issue13985 #13981: time.sleep() should use nanosleep() if available http://bugs.python.org/issue13981 #13979: Automatic *libc.so loading behaviour http://bugs.python.org/issue13979 #13978: OSError exception in multiprocessing module when using os.remo http://bugs.python.org/issue13978 #13977: importlib simplification http://bugs.python.org/issue13977 #13973: urllib.parse is imported twice in xmlrpc.client http:/
Re: [Python-Dev] requirements for moving __import__ over to importlib?
On Thu, Feb 9, 2012 at 17:00, PJ Eby wrote: > On Thu, Feb 9, 2012 at 2:53 PM, Mike Meyer wrote: > >> For those of you not watching -ideas, or ignoring the "Python TIOBE >> -3%" discussion, this would seem to be relevant to any discussion of >> reworking the import mechanism: >> >> http://mail.scipy.org/pipermail/numpy-discussion/2012-January/059801.html >> >> Interesting. This gives me an idea for a way to cut stat calls per > sys.path entry per import by roughly 4x, at the cost of a one-time > directory read per sys.path entry. > > That is, an importer created for a particular directory could, upon first > use, cache a frozenset(listdir()), and the stat().st_mtime of the > directory. All the filename checks could then be performed against the > frozenset, and the st_mtime of the directory only checked once per import, > to verify whether the frozenset() needed refreshing. > I actually contemplated this back in 2006 when I first began importlib for use at Google to get around NFS's crappy stat performance. Never got around to it as compatibility with import.c turned out to be a little tricky. =) Your solution below, PJE, is more-or-less what I was considering (although I also considered variants that didn't stat the directory when you knew your code wasn't changing stuff behind your back). > > Since a failed module lookup takes at least 5 stat checks (pyc, pyo, py, > directory, and compiled extension (pyd/so)), this cuts it down to only 1, > at the price of a listdir(). The big question is how long does a listdir() > take, compared to a stat() or failed open()? That would tell us whether > the tradeoff is worth making. > Actually it's pyc OR pyo, py, directory (which can lead to another set for __init__.py and __pycache__), .so, module.so (or whatever your platform uses for extensions). > > I did some crude timeit tests on frozenset(listdir()) and trapping failed > stat calls. It looks like, for a Windows directory the size of the 2.7 > stdlib, you need about four *failed* import attempts to overcome the > initial caching cost, or about 8 successful bytecode imports. (For Linux, > you might need to double these numbers; my tests showed a different ratio > there, perhaps due to the Linux stdib I tested having nearly twice as many > directory entries as the directory I tested on Windows!) > > However, the numbers are much better for application directories than for > the stdlib, since they are located earlier on sys.path. Every successful > stdlib import in an application is equal to one failed import attempt for > every preceding directory on sys.path, so as long as the average directory > on sys.path isn't vastly larger than the stdlib, and the average > application imports at least four modules from the stdlib (on Windows, or 8 > on Linux), there would be a net performance gain for the application as a > whole. (That is, there'd be an improved per-sys.path entry import time for > stdlib modules, even if not for any application modules.) > Does this comment take into account the number of modules required to load the interpreter to begin with? That's already like 48 modules loaded by Python 3.2 as it is. > > For smaller directories, the tradeoff actually gets better. A directory > one seventh the size of the 2.7 Windows stdlib has a listdir() that's > proportionately faster, but failed stats() in that directory are *not* > proportionately faster; they're only somewhat faster. This means that it > takes fewer failed module lookups to make caching a win - about 2 in this > case, vs. 4 for the stdlib. > > Now, these numbers are with actual disk or network access abstracted away, > because the data's in the operating system cache when I run the tests. > It's possible that this strategy could backfire if you used, say, an NFS > directory with ten thousand files in it as your first sys.path entry. > Without knowing the timings for listdir/stat/failed stat in that setup, > it's hard to say how many stdlib imports you need before you come out > ahead. When I tried a directory about 7 times larger than the stdlib, > creating the frozenset took 10 times as long, but the cost of a failed stat > didn't go up by very much. > > This suggests that there's probably an optimal directory size cutoff for > this trick; if only there were some way to check the size of a directory > without reading it, we could turn off the caching for oversize directories, > and get a major speed boost for everything else. On most platforms, the > stat().st_size of the directory itself will give you some idea, but on > Windows that's always zero. On Windows, we could work around that by using > a lower-level API than listdir() and simply stop reading the directory if > we hit the maximum number of entries we're willing to build a cache for, > and then call it off. > > (Another possibility would be to explicitly enable caching by putting a > flag file in the directory, or perhaps by putting a special prefix on the >
Re: [Python-Dev] PEP 411: Provisional packages in the Python standard library
Other than the misspelling of "maintenante" instead of "maintenance", LGTM. On Fri, Feb 10, 2012 at 09:06, Eli Bendersky wrote: > Hi all, > > Following the intensive and fruitful discussion of the (now rejected) > PEP 408 ( > http://mail.python.org/pipermail/python-dev/2012-January/115850.html), > we've drafted PEP 411 to summarize the conclusions with regards to the > process of marking packages provisional. Note that this is an > informational PEP, and that for the sake of completeness it duplicates > some of the contents of PEP 408. > > It is pasted below, as well as online at > http://www.python.org/dev/peps/pep-0411/. > > Comments are welcome. > > Eli > > > > > PEP: 411 > Title: Provisional packages in the Python standard library > Version: $Revision$ > Last-Modified: $Date$ > Author: Nick Coghlan , >Eli Bendersky > Status: Draft > Type: Informational > Content-Type: text/x-rst > Created: 2012-02-10 > Python-Version: 3.3 > Post-History: 2012-02-10 > > > Abstract > > > The process of including a new package into the Python standard library is > hindered by the API lock-in and promise of backward compatibility implied > by > a package being formally part of Python. This PEP describes a methodology > for marking a standard library package "provisional" for the period of a > single > minor release. A provisional package may have its API modified prior to > "graduating" into a "stable" state. On one hand, this state provides the > package with the benefits of being formally part of the Python > distribution. > On the other hand, the core development team explicitly states that no > promises > are made with regards to the the stability of the package's API, which may > change for the next release. While it is considered an unlikely outcome, > such packages may even be removed from the standard library without a > deprecation period if the concerns regarding their API or maintenante prove > well-founded. > > > Proposal - a documented provisional state > = > > Whenever the Python core development team decides that a new package > should be > included into the standard library, but isn't entirely sure about whether > the > package's API is optimal, the package can be included and marked as > "provisional". > > In the next minor release, the package may either be "graduated" into a > normal > "stable" state in the standard library, or be rejected and removed entirely > from the Python source tree. If the package ends up graduating into the > stable state after being provisional for a minor release, its API may be > changed according to accumulated feedback. The core development team > explicitly makes no guarantees about API stability and backward > compatibility > of provisional packages. > > > Marking a package provisional > - > > A package will be marked provisional by including the following paragraph > as > a note at the top of its documentation page: > >The package has been included in the standard library on a >provisional basis. While major changes are not anticipated, as long as >this notice remains in place, backwards incompatible changes are >permitted if deemed necessary by the standard library developers. Such >changes will not be made gratuitously - they will occur only if >serious API flaws are uncovered that were missed prior to inclusion of >the package. > > Moving a package from the provisional to the stable state simply implies > removing this note from its documentation page. > > > Which packages should go through the provisional state > -- > > We expect most packages proposed for addition into the Python standard > library > to go through a minor release in the provisional state. There may, however, > be some exceptions, such as packages that use a pre-defined API (for > example > ``lzma``, which generally follows the API of the existing ``bz2`` package), > or packages with an API that has wide acceptance in the Python development > community. > > In any case, packages that are proposed to be added to the standard > library, > whether via the provisional state or directly, must fulfill the acceptance > conditions set by PEP 2. > > Criteria for "graduation" > - > > In principle, most provisional packages should eventually graduate to the > stable standard library. Some reasons for not graduating are: > > * The package may prove to be unstable or fragile, without sufficient > developer > support to maintain it. > * A much better alternative package may be found during the preview > release. > > Essentially, the decision will be made by the core developers on a per-case > basis. The point to emphasize here is that a packages's inclusion in the > standard library as "provisional" in some release does not guarantee it > will > continue being part o
Re: [Python-Dev] requirements for moving __import__ over to importlib?
On Fri, Feb 10, 2012 at 1:05 PM, Brett Cannon wrote: > > > On Thu, Feb 9, 2012 at 17:00, PJ Eby wrote: > >> I did some crude timeit tests on frozenset(listdir()) and trapping failed >> stat calls. It looks like, for a Windows directory the size of the 2.7 >> stdlib, you need about four *failed* import attempts to overcome the >> initial caching cost, or about 8 successful bytecode imports. (For Linux, >> you might need to double these numbers; my tests showed a different ratio >> there, perhaps due to the Linux stdib I tested having nearly twice as many >> directory entries as the directory I tested on Windows!) >> > >> However, the numbers are much better for application directories than for >> the stdlib, since they are located earlier on sys.path. Every successful >> stdlib import in an application is equal to one failed import attempt for >> every preceding directory on sys.path, so as long as the average directory >> on sys.path isn't vastly larger than the stdlib, and the average >> application imports at least four modules from the stdlib (on Windows, or 8 >> on Linux), there would be a net performance gain for the application as a >> whole. (That is, there'd be an improved per-sys.path entry import time for >> stdlib modules, even if not for any application modules.) >> > > Does this comment take into account the number of modules required to load > the interpreter to begin with? That's already like 48 modules loaded by > Python 3.2 as it is. > I didn't count those, no. So, if they're loaded from disk *after* importlib is initialized, then they should pay off the cost of caching even fairly large directories that appear earlier on sys.path than the stdlib. We still need to know about NFS and other ratios, though... I still worry that people with more extreme directory sizes or slow-access situations will run into even worse trouble than they have now. > First is that if this were used on Windows or OS X (i.e. the OSs we > support that typically have case-insensitive filesystems), then this > approach would be a massive gain as we already call os.listdir() when > PYTHONCASEOK isn't defined to check case-sensitivity; take your 5 stat > calls and add in 5 listdir() calls and that's what you get on Windows and > OS X right now. Linux doesn't have this check so you would still be > potentially paying a penalty there. > Wow. That means it'd always be a win for pre-stdlib sys.path entries, because any successful stdlib import equals a failed pre-stdlib lookup. (Of course, that's just saving some of the overhead that's been *added* by importlib, not a new gain, but still...) Second is variance in filesystems. Are we guaranteed that the stat of a > directory is updated before a file change is made? > Not quite sure what you mean here. The directory stat is used to ensure that new files haven't been added, old ones removed, or existing ones renamed. Changes to the files themselves shouldn't factor in, should they? > Else there is a small race condition there which would suck. We also have > the issue of granularity; Antoine has already had to add the source file > size to .pyc files in Python 3.3 to combat crappy mtime granularity when > generating bytecode. If we get file mod -> import -> file mod -> import, > are we guaranteed that the second import will know there was a modification > if the first three steps occur fast enough to fit within the granularity of > an mtime value? > Again, I'm not sure how this relates. Automatic code reloaders monitor individual files that have been previously imported, so the directory timestamps aren't relevant. Of course, I could be confused here. Are you saying that if somebody makes a new .py file and saves it, that it'll be possible to import it before it's finished being written? If so, that could happen already, and again caching the directory doesn't make any difference. Alternately, you could have a situation where the file is deleted after we load the listdir(), but in that case the open will fail and we can fall back... heck, we can even force resetting the cache in that event. I was going to say something about __pycache__, but it actually doesn't > affect this. Since you would have to stat the directory anyway, you might > as well just stat directory for the file you want to keep it simple. Only > if you consider __pycache__ to be immutable except for what the interpreter > puts in that directory during execution could you optimize that step (in > which case you can stat the directory once and never care again as the set > would be just updated by import whenever a new .pyc file was written). > > Having said all of this, implementing this idea would be trivial using > importlib if you don't try to optimize the __pycache__ case. It's just a > question of whether people are comfortable with the semantic change to > import. This could also be made into something that was in importlib for > people to use when desired if we are too
[Python-Dev] PEP 411: Provisional packages in the Python standard library
Eli Bendersky wrote (in http://mail.python.org/pipermail/python-dev/2012-February/116393.html ): > A package will be marked provisional by including the > following paragraph as a note at the top of its > documentation page: I really would like some marker available from within Python itself. Use cases: (1) During development, the documentation I normally read first is whatever results from import module; help(module), or possibly dir(module). (2) At BigCorp, there were scheduled times to move as much as possible to the current (or current-1) version. Regardless of policy, full regression test suites don't generally exist. If Python were viewed as part of the infrastructure (rather than as part of a specific application), or if I were responsible for maintaining an internal application built on python, that would be the time to upgrade python -- and I would want an easy way to figure out which applications and libraries I should concentrate on for testing. > * Encapsulation of the import state (PEP 368) Wrong PEP number. I'm guessing that you meant 406. -- If there are still threading problems with my replies, please email me with details, so that I can try to resolve them. -jJ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] requirements for moving __import__ over to importlib?
On Fri, Feb 10, 2012 at 15:07, PJ Eby wrote: > On Fri, Feb 10, 2012 at 1:05 PM, Brett Cannon wrote: > >> >> >> On Thu, Feb 9, 2012 at 17:00, PJ Eby wrote: >> >>> I did some crude timeit tests on frozenset(listdir()) and trapping >>> failed stat calls. It looks like, for a Windows directory the size of the >>> 2.7 stdlib, you need about four *failed* import attempts to overcome the >>> initial caching cost, or about 8 successful bytecode imports. (For Linux, >>> you might need to double these numbers; my tests showed a different ratio >>> there, perhaps due to the Linux stdib I tested having nearly twice as many >>> directory entries as the directory I tested on Windows!) >>> >> >>> However, the numbers are much better for application directories than >>> for the stdlib, since they are located earlier on sys.path. Every >>> successful stdlib import in an application is equal to one failed import >>> attempt for every preceding directory on sys.path, so as long as the >>> average directory on sys.path isn't vastly larger than the stdlib, and the >>> average application imports at least four modules from the stdlib (on >>> Windows, or 8 on Linux), there would be a net performance gain for the >>> application as a whole. (That is, there'd be an improved per-sys.path >>> entry import time for stdlib modules, even if not for any application >>> modules.) >>> >> >> Does this comment take into account the number of modules required to >> load the interpreter to begin with? That's already like 48 modules loaded >> by Python 3.2 as it is. >> > > I didn't count those, no. So, if they're loaded from disk *after* > importlib is initialized, then they should pay off the cost of caching even > fairly large directories that appear earlier on sys.path than the stdlib. > We still need to know about NFS and other ratios, though... I still worry > that people with more extreme directory sizes or slow-access situations > will run into even worse trouble than they have now. > It's possible. No way to make it work for everyone. This is why I didn't worry about some crazy perf optimization. > > > >> First is that if this were used on Windows or OS X (i.e. the OSs we >> support that typically have case-insensitive filesystems), then this >> approach would be a massive gain as we already call os.listdir() when >> PYTHONCASEOK isn't defined to check case-sensitivity; take your 5 stat >> calls and add in 5 listdir() calls and that's what you get on Windows and >> OS X right now. Linux doesn't have this check so you would still be >> potentially paying a penalty there. >> > > Wow. That means it'd always be a win for pre-stdlib sys.path entries, > because any successful stdlib import equals a failed pre-stdlib lookup. > (Of course, that's just saving some of the overhead that's been *added* by > importlib, not a new gain, but still...) > How so? import.c does a listdir() as well (this is not special to importlib). > > > Second is variance in filesystems. Are we guaranteed that the stat of a >> directory is updated before a file change is made? >> > > Not quite sure what you mean here. The directory stat is used to ensure > that new files haven't been added, old ones removed, or existing ones > renamed. Changes to the files themselves shouldn't factor in, should they? > Changes in any fashion to the directory. Do filesystems atomically update the mtime of a directory when they commit a change? Otherwise we have a potential race condition. > > > >> Else there is a small race condition there which would suck. We also have >> the issue of granularity; Antoine has already had to add the source file >> size to .pyc files in Python 3.3 to combat crappy mtime granularity when >> generating bytecode. If we get file mod -> import -> file mod -> import, >> are we guaranteed that the second import will know there was a modification >> if the first three steps occur fast enough to fit within the granularity of >> an mtime value? >> > > Again, I'm not sure how this relates. Automatic code reloaders monitor > individual files that have been previously imported, so the directory > timestamps aren't relevant. > > Don't care about automatic reloaders. I'm just asking about the case where the mtime granularity is coarse enough to allow for a directory change, an import to execute, and then another directory change to occur all within a single mtime increment. That would lead to the set cache to be out of date. > Of course, I could be confused here. Are you saying that if somebody > makes a new .py file and saves it, that it'll be possible to import it > before it's finished being written? If so, that could happen already, and > again caching the directory doesn't make any difference. > > Alternately, you could have a situation where the file is deleted after we > load the listdir(), but in that case the open will fail and we can fall > back... heck, we can even force resetting the cache in that event. > > > I was going to sa
Re: [Python-Dev] requirements for moving __import__ over to importlib?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/10/2012 03:38 PM, Brett Cannon wrote: > Changes in any fashion to the directory. Do filesystems atomically > update the mtime of a directory when they commit a change? Otherwise > we have a potential race condition. Hmm, maybe I misundersand you. In POSIX land, the only thing which changes the mtime of a directory is linking / unlinking / renaming a file: changes to individual files aren't detectable by examining their containing directory's stat(). Tres. - -- === Tres Seaver +1 540-429-0999 [email protected] Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk81jDsACgkQ+gerLs4ltQ7YRwCePFEQA7E74dD9/j8ILuRMHLlA xbkAn1vTYGrEn4VOnVpygGafkGgnm42e =rJGg -END PGP SIGNATURE- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] requirements for moving __import__ over to importlib?
On Fri, Feb 10, 2012 at 16:29, Tres Seaver wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 02/10/2012 03:38 PM, Brett Cannon wrote: > > Changes in any fashion to the directory. Do filesystems atomically > > update the mtime of a directory when they commit a change? Otherwise > > we have a potential race condition. > > Hmm, maybe I misundersand you. In POSIX land, the only thing which > changes the mtime of a directory is linking / unlinking / renaming a > file: changes to individual files aren't detectable by examining their > containing directory's stat(). > Individual file changes are not important; either the module is already in sys.modules so no attempt is made to detect a change or it hasn't been loaded and so it will have to be read regardless. All I'm asking is whether filesystems typically update the filesystem for a e.g. file deletion atomically with the mtime for the containing directory or not. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] requirements for moving __import__ over to importlib?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/10/2012 04:42 PM, Brett Cannon wrote: > On Fri, Feb 10, 2012 at 16:29, Tres Seaver > wrote: > >> On 02/10/2012 03:38 PM, Brett Cannon wrote: >>> Changes in any fashion to the directory. Do filesystems >>> atomically update the mtime of a directory when they commit a >>> change? Otherwise we have a potential race condition. >> >> Hmm, maybe I misundersand you. In POSIX land, the only thing which >> changes the mtime of a directory is linking / unlinking / renaming >> a file: changes to individual files aren't detectable by examining >> their containing directory's stat(). >> > > Individual file changes are not important; either the module is > already in sys.modules so no attempt is made to detect a change or it > hasn't been loaded and so it will have to be read regardless. All I'm > asking is whether filesystems typically update the filesystem for a > e.g. file deletion atomically with the mtime for the containing > directory or not. In POSIX land, most certainly. Tres. - -- === Tres Seaver +1 540-429-0999 [email protected] Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk81kCIACgkQ+gerLs4ltQ5MogCfQwP2n4gl9PfsNXuP3c5al8EX TgwAn2EoGz1vk0OQAh5n3Tl9oze1CSSC =3iuR -END PGP SIGNATURE- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 411: Provisional packages in the Python standard library
On 2/10/2012 9:06 AM, Eli Bendersky wrote: Whenever the Python core development team decides that a new package should be included into the standard library, but isn't entirely sure about whether the package's API is optimal, the package can be included and marked as "provisional". In the next minor release, the package may either be "graduated" into a normal "stable" state in the standard library, or be rejected and removed entirely from the Python source tree. This could be interpreted as limiting provisional status to one release cycle. I suggest that you add 'or continued as provisional'. In particular, if the api *is* changed, another provisional period might be advisable. The package has been included in the standard library on a provisional basis. While major changes are not anticipated, as long as this notice remains in place, backwards incompatible changes are permitted if deemed necessary by the standard library developers. Such 'as long as' implies no particular limit. -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a new "locale" codec?
2012/2/10 "Martin v. Löwis" : >> As And pointed out, this is already the behaviour of the "mbcs" codec >> under Windows. "locale" would be the moral (*) equivalent of that under >> Unix. > > Indeed, and that precedent should be enough reason *not* to include a > "locale" encoding. The "mbcs" encoding has caused much user confusion > over the years, and it is less useful than people typically think. For > example, for some time, people thought that names in zip files ought to > be encoded in "mbcs", only to find out that this is incorrect years > later. With a "locale" encoding, the risk for confusion and untestable > code is too high (just consider the ongoing saga of the Turkish dotless > i (ı)). Well, I expected answer and I agree that there are more drawbacks than advantages. I will close the issue as wontfix. The current locale can already be read using locale.getpreferredencoding(False) and I already fixed functions using the current locale encoding. Victor ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 411: Provisional packages in the Python standard library
On Fri, Feb 10, 2012 at 1:13 PM, Jim J. Jewett wrote: > > Eli Bendersky wrote (in > http://mail.python.org/pipermail/python-dev/2012-February/116393.html ): > >> A package will be marked provisional by including the >> following paragraph as a note at the top of its >> documentation page: > > I really would like some marker available from within Python > itself. > > Use cases: > > (1) During development, the documentation I normally read > first is whatever results from import module; help(module), > or possibly dir(module). > > (2) At BigCorp, there were scheduled times to move as much > as possible to the current (or current-1) version. > Regardless of policy, full regression test suites don't > generally exist. If Python were viewed as part of the > infrastructure (rather than as part of a specific > application), or if I were responsible for maintaining an > internal application built on python, that would be the time > to upgrade python -- and I would want an easy way to figure > out which applications and libraries I should concentrate on > for testing. +1 on both -eric ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] requirements for moving __import__ over to importlib?
On Feb 10, 2012 3:38 PM, "Brett Cannon" wrote: > On Fri, Feb 10, 2012 at 15:07, PJ Eby wrote: >> On Fri, Feb 10, 2012 at 1:05 PM, Brett Cannon wrote: >>> First is that if this were used on Windows or OS X (i.e. the OSs we support that typically have case-insensitive filesystems), then this approach would be a massive gain as we already call os.listdir() when PYTHONCASEOK isn't defined to check case-sensitivity; take your 5 stat calls and add in 5 listdir() calls and that's what you get on Windows and OS X right now. Linux doesn't have this check so you would still be potentially paying a penalty there. >> >> >> Wow. That means it'd always be a win for pre-stdlib sys.path entries, because any successful stdlib import equals a failed pre-stdlib lookup. (Of course, that's just saving some of the overhead that's been *added* by importlib, not a new gain, but still...) > > > How so? import.c does a listdir() as well (this is not special to importlib). IIRC, it does a FindFirstFile on Windows, which is not the same thing. That's one system call into a preallocated buffer, not a series of system calls and creation of Python string objects. > Don't care about automatic reloaders. I'm just asking about the case where the mtime granularity is coarse enough to allow for a directory change, an import to execute, and then another directory change to occur all within a single mtime increment. That would lead to the set cache to be out of date. Ah. Good point. Well, if there's any way to know what the mtime granularity is, we can avoid the race condition by never performing the listdir when the current clock time is too close to the stat(). In effect, we can bypass the optimization if the directory was just modified. Something like: mtime = stat(dir).st_mtime if abs(time.time()-mtime)>unsafe_window: old_mtime, files = cache.get(dir, (-1, ())) if mtime!=old_mtime: files = frozenset(listdir(dir)) cache[dir] = mtime, files # code to check for possibility of importing # and shortcut if found, or # exit with failure if no matching files # fallthrough to direct filesystem checking The "unsafe window" is presumably filesystem and platform dependent, but ISTR that even FAT filesystems have 2-second accuracy. The other catch is the relationship between st_mtime and time.time(); I assume they'd be the same in any sane system, but what if you're working across a network and there's clock skew? Ugh. Worst case example would be say, accessing a FAT device that's been shared over a Windows network from a machine whose clock is several hours off. So it always looks safe to read, even if it's just been changed. What's the downside in that case? You're trying to import something that just changed in the last fraction of a second... why? I mean, sure, the directory listing will be wrong, no question. But it only matters that it was wrong if you added, removed, or renamed importable files. Why are you trying to import one of them? Ah, here's a use case: you're starting up IDLE, and while it's loading, you save some .py files you plan to import later. Your editor saves them all at once, but IDLE does the listdir() midway through. You then do an import from the IDLE prompt, and it fails because the listdir() didn't catch everything. Okay, now I know how to fix this. The problem isn't that there's a race condition per se, the problem is that the race results in a broken cache later. After all, it could just as easily have been the case that the import failed due to timing. The problem is that all *future* imports would fail in this circumstance. So the fix is a time-to-live recheck: if TTL seconds have passed since the last use of the cached frozenset, reload it, and reset the TTL to infinity. In other words: mtime = stat(dir).st_mtime now - time.time() if abs(now-mtime)>unsafe_window: old_mtime, then, files = cache.get(dir, (-1, now, ())) if mtime!=old_mtime or then is not None and now-then>TTL: files = frozenset(listdir(dir)) cache[dir] = mtime, now if mtime!=old_mtime else None, files # code to check for possibility of importing # and shortcut if found, or # exit with failure if no matching files # fallthrough to direct filesystem checking What this does (or should do) is handle clock-skew race condition stale caches by reloading the listdir even if mtime hasn't changed, as soon as TTL seconds have passed since the last snapshot was taken. However, if the mtime stays the same, no subsequent listdirs will occur. As long as the TTL is set high enough that a full startup of Python can occur, but low enough that it resets by the time a human can notice something's wrong, it should be golden. ;-) The TTL approach could be used in place of the unsafe_window, actually; there's probably not much need for both. The pure unsafe_window a
Re: [Python-Dev] requirements for moving __import__ over to importlib?
On Sat, Feb 11, 2012 at 11:23 AM, PJ Eby wrote: > What's the downside in that case? You're trying to import something that > just changed in the last fraction of a second... why? I don't know if it's normal in the Python world, but these sorts of race conditions occur most annoyingly when a single process changes a file, then attempts to import it. If you open a file, write to it, explicitly close it, and then load it, you would expect to read back what you wrote, not the version that was there previously. Chris Angelico ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] http://pythonmentors.com/
Hi all, I'd never heard of this until some Dutch geezer whose name I'm now forgotten pointed me to it. Had I known about it a couple of years ago it would have saved a lot of people a lot of grief. Please could it be given a bit of publicity. -- Cheers. Mark Lawrence. p.s. The Dutch geezer in question competes with Dr. Who for having the bast time travelling machine :) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] http://pythonmentors.com/
I've been trying to publicize it on twitter, my blog, google plus and elsewhere. help welcome. On Friday, February 10, 2012 at 8:27 PM, Mark Lawrence wrote: > Hi all, > > I'd never heard of this until some Dutch geezer whose name I'm now > forgotten pointed me to it. Had I known about it a couple of years ago > it would have saved a lot of people a lot of grief. Please could it be > given a bit of publicity. > > -- > Cheers. > > Mark Lawrence. > > p.s. The Dutch geezer in question competes with Dr. Who for having the > bast time travelling machine :) > > ___ > Python-Dev mailing list > [email protected] (mailto:[email protected]) > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 411: Provisional packages in the Python standard library
Jim J. Jewett wrote: Eli Bendersky wrote (in http://mail.python.org/pipermail/python-dev/2012-February/116393.html ): A package will be marked provisional by including the following paragraph as a note at the top of its documentation page: I really would like some marker available from within Python itself. +1 -- Steven ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Fwd: maintenance of the ElementTree / cElementTree packages in the Python standard library
A protagonist writes: > -- Forwarded message -- > From: Fredrik Lundh As a not-directly-concerned third party who's been lurking, it seems to me like people are in way too much of a rush to "get things done". Sending direct mail, addressing the precise question[1] seems to have produced immediate results. (And yes, I've been on this list long enough to know that maintenance of ET/cET has been an issue for years. Nevertheless!) While actually maintaining the code is important, continuity in the community is important, too, and probably more so. In this case, continuity evidently will be achieved by a handoff rather than reactivation of Fredrik. It's a shame that this result had to be achieved by MvL putting his foot down. I'm glad he did. Footnotes: [1] Ie, not "when is this or that code issue going to be addressed", but "in view of your apparent schedule constraints, would it be OK to maintain the package in the stdlib with python-dev taking responsibility". ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 411: Provisional packages in the Python standard library
On Fri, Feb 10, 2012 at 20:33, Brett Cannon wrote: > Other than the misspelling of "maintenante" instead of "maintenance", LGTM. > Fixed that and another typo (thanks 'aspell' :-] ) Eli ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 411: Provisional packages in the Python standard library
On Fri, Feb 10, 2012 at 22:13, Jim J. Jewett wrote: > > Eli Bendersky wrote (in > http://mail.python.org/pipermail/python-dev/2012-February/116393.html ): > >> A package will be marked provisional by including the >> following paragraph as a note at the top of its >> documentation page: > > I really would like some marker available from within Python > itself. > > Use cases: > > (1) During development, the documentation I normally read > first is whatever results from import module; help(module), > or possibly dir(module). > > (2) At BigCorp, there were scheduled times to move as much > as possible to the current (or current-1) version. > Regardless of policy, full regression test suites don't > generally exist. If Python were viewed as part of the > infrastructure (rather than as part of a specific > application), or if I were responsible for maintaining an > internal application built on python, that would be the time > to upgrade python -- and I would want an easy way to figure > out which applications and libraries I should concentrate on > for testing. > The big problem with this is that it's something that will have to be maintained, so it adds some additional burden (I suppose it will have to be tested as well). An easy way for (2) would be just grepping on the Python docs for the provisional note and seeing which modules have it. Anyhow, I'm not against the idea. I just think it has to be discussed in more detail so all the implications are understood. >> * Encapsulation of the import state (PEP 368) > > Wrong PEP number. I'm guessing that you meant 406. > Yep, thanks. Fixed. Eli ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 411: Provisional packages in the Python standard library
On Fri, Feb 10, 2012 at 23:56, Terry Reedy wrote: > On 2/10/2012 9:06 AM, Eli Bendersky wrote: > >> Whenever the Python core development team decides that a new package >> should be >> included into the standard library, but isn't entirely sure about whether >> the >> package's API is optimal, the package can be included and marked as >> "provisional". >> >> In the next minor release, the package may either be "graduated" into a >> normal >> "stable" state in the standard library, or be rejected and removed >> entirely >> from the Python source tree. > > > This could be interpreted as limiting provisional status to one release > cycle. I suggest that you add 'or continued as provisional'. In particular, > if the api *is* changed, another provisional period might be advisable. > I think this was agreed upon when PEP 408 was discussed. Keeping a package provisional for too long is detrimental. Isn't a single release enough to decide that we want something or not? Keep in mind that many users won't touch the provisional packages in production code - we would like to make new parts of the stdlib functional as soon as possible. > >> The package has been included in the standard library on a >> provisional basis. While major changes are not anticipated, as long as >> this notice remains in place, backwards incompatible changes are >> permitted if deemed necessary by the standard library developers. Such > > > 'as long as' implies no particular limit. Perhaps it should also? Eli ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] http://pythonmentors.com/
On Sat, Feb 11, 2012 at 03:38, Jesse Noller wrote: > I've been trying to publicize it on twitter, my blog, google plus and > elsewhere. > > help welcome. > It also appears in the first paragraph of "Contributing" in the dev guide - which is pointed to by the main page at python.org (Core Development link). Mark, do you have a concrete idea of how it can be made more prominent? Eli ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 411: Provisional packages in the Python standard library
On Fri, Feb 10, 2012 at 8:10 PM, Eli Bendersky wrote: > On Fri, Feb 10, 2012 at 22:13, Jim J. Jewett wrote: >> >> Eli Bendersky wrote (in >> http://mail.python.org/pipermail/python-dev/2012-February/116393.html ): >> >>> A package will be marked provisional by including the >>> following paragraph as a note at the top of its >>> documentation page: >> >> I really would like some marker available from within Python >> itself. >> > > The big problem with this is that it's something that will have to be > maintained, so it adds some additional burden (I suppose it will have > to be tested as well). > > An easy way for (2) would be just grepping on the Python docs for the > provisional note and seeing which modules have it. > > Anyhow, I'm not against the idea. I just think it has to be discussed > in more detail so all the implications are understood. Is there more to it than having a simple __provisional__ attribute on the module and/or a list at sys.provisional_modules? -eric ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] http://pythonmentors.com/
On Sat, Feb 11, 2012 at 1:14 PM, Eli Bendersky wrote: > Mark, do you have a concrete idea of how it can be made more prominent? Mark didn't know about it because the core-mentorship list didn't exist yet in the timeframe he's talking about :) Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 411: Provisional packages in the Python standard library
On Sat, Feb 11, 2012 at 1:39 PM, Eric Snow wrote: > Is there more to it than having a simple __provisional__ attribute on > the module and/or a list at sys.provisional_modules? Yes. As soon as we touch functional code, it because something to be tested and the process overhead on our end is noticeably higher. However, I'd be fine with requiring that a short form for the notice appear at the start of the module docstring. For example: "The API of this module is currently provisional. Refer to the documentation for details." This would then be seen by pydoc and help(), as well as being amenable to programmatic inspection. Also, with a documented provisional status, I think keeping things provisional for as long as it takes us to make up our minds they're right is fine. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 411: Provisional packages in the Python standard library
Eric Snow wrote: On Fri, Feb 10, 2012 at 8:10 PM, Eli Bendersky wrote: On Fri, Feb 10, 2012 at 22:13, Jim J. Jewett wrote: Eli Bendersky wrote (in http://mail.python.org/pipermail/python-dev/2012-February/116393.html ): A package will be marked provisional by including the following paragraph as a note at the top of its documentation page: I really would like some marker available from within Python itself. The big problem with this is that it's something that will have to be maintained, so it adds some additional burden (I suppose it will have to be tested as well). "Big problem"? Maintenance of bsddb3 has been a big problem. Maintenance of a single module-level name for provisional packages is a small problem. The PEP already gives boilerplate which is required to go into the documentation of provisional packages. Requiring a top level name, and test for that, is no harder than what's already expected, and it is a constant difficulty regardless of package. In fact, we could (should?) have a single test that applies to all packages in the std lib: for package in packages: if isprovisional(package): assert hasattr(package, '__provisional__') assert package documentation includes boilerplate else: assert not hasattr(package, '__provisional__') assert package documentation does not include boilerplate Arguably, the canonical test for whether a package is provisional or not should be the existence of __provisional__: for package in packages: if hasattr(package, '__provisional__') assert package documentation includes boilerplate else: assert package documentation does not includes boilerplate An easy way for (2) would be just grepping on the Python docs for the provisional note and seeing which modules have it. Not all OSes include grep. Not all Python installations include the docs. -- Steven ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] http://pythonmentors.com/
On Sat, Feb 11, 2012 at 08:27, Nick Coghlan wrote: > On Sat, Feb 11, 2012 at 1:14 PM, Eli Bendersky wrote: >> Mark, do you have a concrete idea of how it can be made more prominent? > > Mark didn't know about it because the core-mentorship list didn't > exist yet in the timeframe he's talking about :) > Yes, but he *now* asks to give it more publicity. Hence my question. Eli ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
