Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Sat, Jan 28, 2012 at 9:49 AM, Matt Joiner wrote: > FWIW I'm now -1 for this idea. Stronger integration with PyPI and > packaging systems is much preferable. Python core public releases are > no place for testing. +1. I'd much rather just use the module from PyPI. It would be good to have a practical guide on how to manage the transition from third-party to core library module though. A PEP with a list of modules earmarked for upcoming inclusion in the standard library (and which Python version they're intended to be included in) might focus community effort on using, testing and fixing modules before they make it into core and fixing becomes a lot harder. Schiavo Simon ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
> +1. I'd much rather just use the module from PyPI. > > It would be good to have a practical guide on how to manage the > transition from third-party to core library module though. A PEP with > a list of modules earmarked for upcoming inclusion in the standard > library (and which Python version they're intended to be included in) > might focus community effort on using, testing and fixing modules > before they make it into core and fixing becomes a lot harder. +1 for your +1, and earmarking. That's the word I was looking for, and instead chose "advocacy". ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
Nick Coghlan writes: > People need to remember there's another half to this equation: the > core dev side. Why? There's nothing about it in the PEP. > The reason *regex* specifically isn't in the stdlib already is > largely due to (perhaps excessive) concerns about the potential > maintenance burden. But then giving regex as an example seems to contradict the PEP: "The only difference between preview APIs and the rest of the standard library is that preview APIs are explicitly exempted from the usual backward compatibility guarantees," "in principle, most modules in the __preview__ package should eventually graduate to the stable standard library," and "whenever the Python core development team decides that a new module should be included into the standard library, but isn't sure about whether the module's API is optimal". True, there were a few bits spilled on the possibility of being "without sufficient developer support to maintain it," but I read that as a risk that is basically a consequence of instability of the API. The rationale is entirely focused on API instability, and a focus on API instability is certainly the reason for calling it "__preview__" rather than "__experimental__". I don't have an opinion on whether this is an argument for rejecting the PEP or for rewriting it (specifically, seriously beefing up the "after trying it, maybe we won't want to maintain it" rationale). I also think that if "we need to try it to decide if the maintenance burden is acceptable" is a rationale, the name "__experimental__" should be seriously reconsidered as more accurately reflecting the intended content of the package. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Sat, Jan 28, 2012 at 5:49 PM, Matt Joiner wrote: > FWIW I'm now -1 for this idea. Stronger integration with PyPI and > packaging systems is much preferable. Python core public releases are > no place for testing. People saying this: we KNOW this approach doesn't work in all cases. If it worked perfectly, regex would be in the standard library by now. Don't consider this PEP a purely theoretical proposal, because it isn't. It's really being put forward to solve a specific problem: the fact that we need to do something about re's lack of proper Unicode support [1]. Those issues are actually hard to solve, so replacing re with Matthew Barnett's regex module (just as re itself was a replacement for the original regex module) that already addresses most of them seems like a good way forward, but this is currently being blocked because there are still a few lingering concerns with maintainability and backwards compatibility. We *need* to break the impasse preventing its inclusion in the standard library, and __preview__ lets us do that without running roughshod over the legitimate core developer concerns raised in the associated tracker issue [2]. With the current criteria for stdlib inclusion, it doesn't *matter* if a module is oh-so-close to being accepted: it gets rejected anyway, just like a module that has no chance of ever being suitable. There is currently *no* path forward for resolving any stdlib-specific concerns that arise with already popular PyPI modules, and so such situations remain unresolved and key components of the standard library stagnate. While regex is the current poster-child for this problem, it's quite likely that similar problems will arise in the future. Kenneth Reitz's requests module is an obvious candidate: it's enormously popular with users, Kenneth has indicated he's amenable to the idea of stdlib inclusion once the feature set is sufficiently stable (i.e. not for 3.3), but I expect there will be legitimate concerns with incorporating it, given its scope. Cheers, Nick. [1] http://bugs.python.org/issue?%40search_text=&ignore=file%3Acontent&title=&%40columns=title&id=&%40columns=id&stage=&creation=&creator=tchrist&activity=&%40columns=activity&%40sort=activity&actor=&nosy=&type=&components=&versions=&dependencies=&assignee=&keywords=&priority=&%40group=priority&status=1&%40columns=status&resolution=&nosy_count=&message_count=&%40pagesize=50&%40startwith=0&%40queryname=&%40old-queryname=&%40action=search [2] http://bugs.python.org/issue2636 -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Sat, Jan 28, 2012 at 6:38 PM, Stephen J. Turnbull wrote: > I don't have an opinion on whether this is an argument for rejecting > the PEP or for rewriting it (specifically, seriously beefing up the > "after trying it, maybe we won't want to maintain it" rationale). I > also think that if "we need to try it to decide if the maintenance > burden is acceptable" is a rationale, the name "__experimental__" > should be seriously reconsidered as more accurately reflecting the > intended content of the package. I think it's an argument for rewriting it (and, as you point out, perhaps reverting to __experimental__ as the proposed name). Eli started from a draft I wrote a while back and my own thinking on the topic wasn't particularly clear (in fact, it's only this thread that has really clarified things for me). The main thing I've realised is that the end user benefits currently discussed in the PEP are really about the importance of a robust *standard library*. They aren't specific to the new namespace at all - that part of the rationale is really only needed to counter the predictable "who cares about the standard library, we can just use PyPI!" responses (and the answer is, "lots of people that can't or won't use PyPI modules for a wide range of reasons"). The only reason to add a new double-underscore namespace is to address *core developer* concerns in cases where we're *almost* sure that we want to add the module to the standard library, but aren't quite prepared to commit to maintaining it for the life of the 3.x series (cf. 2.x and the ongoing problems we had with keeping the bsddb module working properly, especially before Jesus Cea stepped up to wrangle it into submission). It's basically us saying to Python users "We're explicitly flagging this PyPI module for inclusion in the next major Python release. We've integrated it into our build process, test suite and binary releases, so you don't even have to download it from PyPI in order to try it out, you can just import it from the __preview__ namespace (although you're still free to download it from PyPI if you prefer - in fact, if you need to support multiple Python versions, we actively recommend it!). There's still a small chance this module won't make the grade and will be dropped from the standard library entirely (that's why it's only a preview), but most likely it will move into the main part of the standard library with full backwards compatibility guarantees in the next release". Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3 optimizations, continued, continued again...
stefan brunthaler wrote: Hi, On Tue, Nov 8, 2011 at 10:36, Benjamin Peterson wrote: 2011/11/8 stefan brunthaler : How does that sound? I think I can hear real patches and benchmarks most clearly. I spent the better part of my -20% time on implementing the work as "suggested". Please find the benchmarks attached to this email, I just Could you try benchmarking with the "standard" benchmarks: http://hg.python.org/benchmarks/ and see what sort of performance gains you get? did them on my system (i7-920, Linux 3.0.0-15, GCC 4.6.1). I branched off the regular 3.3a0 default tip changeset 73977 shortly after your email. I do not have an official patch yet, but am going to create one if wanted. Changes to the existing interpreter are minimal, the biggest chunk is a new interpreter dispatch loop. How portable is the threaded interpreter? Do you have a public repository for the code, so we can take a look? Cheers, Mark. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
Wiadomość napisana przez Simon Cross w dniu 28 sty 2012, o godz. 08:58: > +1. I'd much rather just use the module from PyPI. > > It would be good to have a practical guide on how to manage the > transition from third-party to core library module though. A PEP with > a list of modules earmarked for upcoming inclusion in the standard > library (and which Python version they're intended to be included in) > might focus community effort on using, testing and fixing modules > before they make it into core and fixing becomes a lot harder. +1 -- Best regards, Łukasz Langa Senior Systems Architecture Engineer IT Infrastructure Department Grupa Allegro Sp. z o.o. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python 3 benchmarks
Hi Something that's maybe worth mentioning is that the "official" python benchmark suite http://hg.python.org/benchmarks/ has a pretty incomplete set of benchmarks for python 3 compared to say what we run for pypy: https://bitbucket.org/pypy/benchmarks I think a very worthwhile project would be to try to port other benchmarks (that actually use existing python projects like sympy or django) for those that has been ported to python 3. Any thoughts? Cheers, fijal ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On 28 January 2012 09:18, Nick Coghlan wrote: > It's basically us saying to Python users "We're explicitly flagging > this PyPI module for inclusion in the next major Python release. We've > integrated it into our build process, test suite and binary releases, > so you don't even have to download it from PyPI in order to try it > out, you can just import it from the __preview__ namespace (although > you're still free to download it from PyPI if you prefer - in fact, if > you need to support multiple Python versions, we actively recommend > it!). There's still a small chance this module won't make the grade > and will be dropped from the standard library entirely (that's why > it's only a preview), but most likely it will move into the main part > of the standard library with full backwards compatibility guarantees > in the next release". +1. Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On 28 January 2012 01:48, Barry Warsaw wrote: > The thinking goes like this: if you would normally use an __preview__ module > because you can't get approval to download some random package from PyPI, well > then your distro probably could or should provide it, so get it from them. In > fact, if the number of __preview__ modules is kept low, *and* PyPI equivalents > were a requirement, then a distro vendor could just ensure those PyPI versions > are available as distro packages outside of the __preview__ stdlib namespace > (i.e. in their normal third-party namespace). Then folks developing on that > platform could just use the distro package and ignore __preview__. Just so that you know that such cases exist, I am in a position where I have access to systems with (distro-supplied) Python installed. I can use anything supplied with Python (i.e., the stdlib - and __preview__ would fall into this category as well). And yet I have essentially no means of gaining access to any 3rd party modules, whether they are packaged by the distro or obtained from PyPI. (And "build your own" isn't an option in many cases, if only because a C compiler may well not be available!) This is essentially due to corporate inertia and bogged down "do-nothing" policies rather than due dilligence or supportability concerns. But it is a reality for me (and many others, I suspect). Having said this, of course, the same corporate inertia means that Python 3.3 is a pipe-dream for me in those environments for many years yet. So ignoring them may be reasonable. Just some facts to consider :-) Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On 1/28/2012 2:10 AM, Nick Coghlan wrote: > On Sat, Jan 28, 2012 at 3:22 PM, Stephen J. Turnbull > wrote: >> Executive summary: >> >> If the promise to remove the module from __preview__ is credible (ie, >> strictly kept), then __preview__ will have a specific audience in >> those who want the stdlib candidate code and are willing to deal with >> a certain amount of instability in that code. > > People need to remember there's another half to this equation: the > core dev side. > > The reason *regex* specifically isn't in the stdlib already is largely > due to (perhaps excessive) concerns about the potential maintenance > burden. It's not a small chunk of code and we don't want to deal with > another bsddb. ... > Really, the main benefit for end users doesn't lie in __preview__ > itself: it lies in the positive effect __preview__ will have on the > long term evolution of the standard library, as it aims to turn > python-dev's inherent conservatism (which is a good thing!) into a > speed bump rather than a road block. I was -0 on this proposal, but after Nick's discussion above I'm now +1. I also think it's worth thinking about how multiprocessing would have benefited from the __preview__ process. And for people saying "just use PyPI": that tends to exclude many Windows users from trying out packages that aren't pure Python. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
> __preview__ would fall into this category as well). And yet I have > essentially no means of gaining access to any 3rd party modules, > whether they are packaged by the distro or obtained from PyPI. (And > "build your own" isn't an option in many cases, if only because a C > compiler may well not be available!) This is essentially due to > corporate inertia and bogged down "do-nothing" policies rather than > due dilligence or supportability concerns. But it is a reality for me > (and many others, I suspect). > > Having said this, of course, the same corporate inertia means that > Python 3.3 is a pipe-dream for me in those environments for many years > yet. So ignoring them may be reasonable. You clearly want access to external modules sooner. A preview namespace addresses this indirectly. The separated stdlib versioning concept is far superior for this use case. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
Hi, Am 27.01.2012 um 18:26 schrieb Alex: > I'm -1 on this, for a pretty simple reason. Something goes into __preview__, > instead of it's final destination directly because it needs feedback/possibly > changes. However, given the release cycle of the stdlib (~18 months), any > feedback it gets can't be seen by actual users until it's too late. > Essentially > you can only get one round of stdlib. > > I think a significantly healthier process (in terms of maximizing feedback and > getting something into it's best shape) is to let a project evolve naturally > on > PyPi and in the ecosystem, give feedback to it from an inclusion perspective, > and then include it when it becomes ready on it's own merits. The counter > argument to this is that putting it in the stdlib gets you signficantly more > eyeballs (and hopefully more feedback, therefore), my only response to this > is: > if it doesn't get eyeballs on PyPi I don't think there's a great enough need > to > justify it in the stdlib. I agree with Alex on this: The iterations – even with PEP 407 – would be wayyy too long to be useful. As for the only downside: How about endorsing certain pypi projects as possible future additions in order to give them more exposure? I'm sure there is some nice way for that. Plus: Everybody could pin the version their code depends on right now, so updates wouldn't break anything. I.e. api users would have more peace of mind and api developers could develop more aggressively. Bye, -h ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On 28/01/2012 13:04, Paul Moore wrote: On 28 January 2012 09:18, Nick Coghlan wrote: It's basically us saying to Python users "We're explicitly flagging this PyPI module for inclusion in the next major Python release. We've integrated it into our build process, test suite and binary releases, so you don't even have to download it from PyPI in order to try it out, you can just import it from the __preview__ namespace (although you're still free to download it from PyPI if you prefer - in fact, if you need to support multiple Python versions, we actively recommend it!). There's still a small chance this module won't make the grade and will be dropped from the standard library entirely (that's why it's only a preview), but most likely it will move into the main part of the standard library with full backwards compatibility guarantees in the next release". +1. Yep, nice way of putting it - and summing up the virtues of the approach. (Although I might say "most likely it will move into the main part of the standard library with full backwards compatibility guarantees in a future release".) Michael Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On 27/01/2012 22:54, Barry Warsaw wrote: On Jan 27, 2012, at 10:48 PM, Antoine Pitrou wrote: On Fri, 27 Jan 2012 16:10:51 -0500 Barry Warsaw wrote: I'm -1 on this as well. It just feels like the completely wrong way to stabilize an API, and I think despite the caveats that are explicit in __preview__, Python will just catch tons of grief from users and haters about API instability anyway, because from a practical standpoint, applications written using __preview__ APIs *will* be less stable. Well, obviously __preview__ is not for the most conservative users. I think the name clearly conveys the idea that you are trying out something which is not in its definitive state, doesn't it? Maybe. I could quibble about the name, but let's not bikeshed on that right now. The problem as I see it is that __preview__ will be very tempting to use in production. In fact, its use case is almost predicated on that. (We want you to use it so you can tell us if the API is good.) Once people use it, they will probably ship code that relies on it, and then the pressure will be applied to us to continue to support that API even if a newer, better one gets promoted out of __preview__. I worry that over time, for all practical purposes, there won't be much difference between __preview__ and the stdlib. I think a significantly healthier process (in terms of maximizing feedback and getting something into it's best shape) is to let a project evolve naturally on PyPi and in the ecosystem, give feedback to it from an inclusion perspective, and then include it when it becomes ready on it's own merits. The counter argument to this is that putting it in the stdlib gets you signficantly more eyeballs (and hopefully more feedback, therefore), my only response to this is: if it doesn't get eyeballs on PyPi I don't think there's a great enough need to justify it in the stdlib. I agree with everything Alex said here. The idea that being on PyPI is sufficient is nice but flawed (the IPaddr example). PyPI doesn't guarantee any visibility (how many packages are there?). Furthermore, having users is not a guarantee that the API is appropriate, either; it just means that the API is appropriate for *some* users. I can't argue with that, it's just that I don't think __preview__ solves that problem. And it seems to me that __preview__ introduces a whole 'nother set of problems on top of that. So taking the IPaddr example further. Would having it in the stdlib, relegated to an explicitly unstable API part of the stdlib, increase eyeballs enough to generate the kind of API feedback we're looking for, without imposing an additional maintenance burden on us? I think the answer is yes. That's kind of the crux of the matter I guess. If you were writing an app that used something in __preview__, how would you provide feedback on what parts of the API you'd want to change, The bugtracker. *and* how would you adapt your application to use those better APIs once they became available 18 months from now? How do users do it for the standard library? Using the third party version is one way. I think we'll just see folks using the unstable APIs and then complaining when we remove them, even though they *know* *upfront* that these APIs will go away. I'm also nervous about it from an OS vender point of view. Should I reject any applications that import from __preview__? Or do I have to make a commitment to support those APIs longer than Python does because the application that uses it is important to me? I think the OS vendor problem is easier with an application that uses some PyPI package, because I can always make that package available to the application by pulling in the version I care about. It's harder if a newer, incompatible version is released upstream and I want to provide both, but I don't think __preview__ addresses that. A robust, standard approach to versioning of modules would though, and I think would better solve what __preview__ is trying to solve. Don't OS vendors go further and say "pin your dependency to the version we ship", whether it's in the Python standard library or not? So "just use a more recent version from pypi" is explicitly not an option for people using system packages. As OS packagers tend to target a specific version of python, using __preview__ for that version would be fine - and when they upgrade to the next version applications may need fixing in the same way as they would if the system packaged a new release of the third party library. (When moving between Ubuntu distributions I've found that my software using system packages often needs to change because the version of some library has now changed.) Plus having a package in __preview__ has no bearing on whether or not the system packages the third party version, so I think it's a bit of a red-herring. Michael On the other hand, __preview__ would clearly signal that something is on the verge of being frozen as an
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On 28/01/2012 05:09, Scott Dial wrote: On 1/27/2012 8:48 PM, Barry Warsaw wrote: The thinking goes like this: if you would normally use an __preview__ module because you can't get approval to download some random package from PyPI, well then your distro probably could or should provide it, so get it from them. That is my thought about the entire __preview__ concept. Anything that would/should go into __preview__ would be better off being packaged for a couple of key distros (e.g., Ubuntu/Fedora/Gentoo) where they would get better visibility than just being on PyPI and would be more flexible in terms of release schedule to allow API changes. If the effort being put into making the __preview__ package was put into packaging those modules for distros, That effort wouldn't be put in though. Largely those involved in working on Python are not the ones packaging for Linux distributions. So it isn't an alternative to __preview__ - it could happily be done alongside it though. Those who work on Python won't just switch to Linux if this proposal isn't accepted, they'll do different work on Python instead. then you would get the same exposure Packaging libraries for Linux gets you no exposure on Windows or the Mac, so __preview__ is wider. with better flexibility and a better maintenance story. The whole idea of __preview__ seems to be a workaround for the difficult packaging story for Python modules on common distros I don't know where you got that impression. :-) One of the reasons for __preview__ is that it means integrating libraries with the Python build and test systems, for all platforms. Packaging for [some-variants-of] Linux only doesn't do anything for this. All the best, Michael -- stuffing them into __preview__ is a cheat to get the distro packagers to distribute these interesting modules since we would be bundling them. However, as you have pointed out, it would very desirable to them to not do so. So in the end, these modules may not receive as wide of visibility as the PEP suggests. I could very easily imagine the more stable distributions refusing or patching anything that used __preview__ in order to eliminate difficulties. -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On 28/01/2012 13:55, Matt Joiner wrote: __preview__ would fall into this category as well). And yet I have essentially no means of gaining access to any 3rd party modules, whether they are packaged by the distro or obtained from PyPI. (And "build your own" isn't an option in many cases, if only because a C compiler may well not be available!) This is essentially due to corporate inertia and bogged down "do-nothing" policies rather than due dilligence or supportability concerns. But it is a reality for me (and many others, I suspect). Having said this, of course, the same corporate inertia means that Python 3.3 is a pipe-dream for me in those environments for many years yet. So ignoring them may be reasonable. You clearly want access to external modules sooner. A preview namespace addresses this indirectly. The separated stdlib versioning concept is far superior for this use case. There are two proposals for the standard library - one is to do development in a separate repository to make it easier for other implementations to contribute. To my understanding this proposal is mildly controversial, but doesn't involve changing the way the standard library is distributed or versioned. A separate proposal about standard library versioning has been floated but is *much* more controversial and therefore much less likely to happen. So I wouldn't hold your breath on it... All the best, Michael Foord ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On 28/01/2012 04:44, Stephen J. Turnbull wrote: Michael Foord writes: > >> Assuming the module is then promoted to the the standard library proper in > >> release ``3.X+1``, it will be moved to a permanent location in the library:: > >> > >> import example > >> > >> And importing it from ``__preview__`` will no longer work. > > Why not leave it accessible through __preview__ too? > > +1 Er, doesn't this contradict your point about using try: from __preview__ import spam except ImportError: import spam ? I think it's a bad idea to introduce a feature that's *supposed* to break (in the sense of "make a break", ie, change the normal pattern) with every release and then try to avoid breaking (in the sense of "causing an unexpected failure") code written by people who don't want to follow the discipline of keeping up with changing APIs. If they want that stability, they should wait for the stable release. Modules should become unavailable from __preview__ as soon as they have a stable home. I like not breaking people's code where *possible*. Michael -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Hashing proposal: 64-bit hash
27.01.12 23:08, Frank Sievertsen написав(ла): As already mentioned, the vulnerability of 64-bit Python rather theoretical and not practical. The size of the hash makes the attack is extremely unlikely. Unfortunately this assumption is not correct. It works very good with 64bit-hashing. It's much harder to create (efficiently) 64-bit hash-collisions. But I managed to do so and created strings with a length of 16 (6-bit)-characters (a-z, A-Z, 0-9, _, .). Even 14 characters would have been enough. You need less than twice as many characters for the same effect as in the 32bit-world. The point is not the length of the string, but the size of string space for inspection. To search for a string with a specified 64-bit hash to iterate over 2 ** 64 strings. Spending on a single string scan 1 nanosecond (a very optimistic estimate), it would take 2 ** 64 / 1e9 / (3600 * 24 * 365.25) = 585 years. For the attack we need to find 1000 such strings -- more than half a million years. For 32-bit hash would need only an hour. Of course, to calculate the hash function to use secure, not allowing "cut corners" and reduce computation time. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Sat, 28 Jan 2012 02:49:40 -0500 Matt Joiner wrote: > FWIW I'm now -1 for this idea. Stronger integration with PyPI and > packaging systems is much preferable. That will probably never happen. "pip install XXX" is the best we (python-dev and the community) can do. "import some_module" won't magically start fetching some_module from PyPI if it isn't installed on your system. So the bottom line is: we would benefit from an intermediate status between "available on PyPI" and "shipped as a stable API in the stdlib". The __preview__ proposal does just that in an useful way; are there any alternatives you'd like to suggest? Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Sat, 28 Jan 2012 00:09:13 -0500 Scott Dial wrote: > On 1/27/2012 8:48 PM, Barry Warsaw wrote: > > The thinking goes like this: if you would normally use an __preview__ module > > because you can't get approval to download some random package from PyPI, > > well > > then your distro probably could or should provide it, so get it from them. > > That is my thought about the entire __preview__ concept. Anything that > would/should go into __preview__ would be better off being packaged for > a couple of key distros (e.g., Ubuntu/Fedora/Gentoo) where they would > get better visibility than just being on PyPI and would be more flexible > in terms of release schedule to allow API changes. This is a red herring. First, not everyone uses a distro. There are almost a million monthly downloads of the Windows installers. Second, what a distro puts in their packages has nothing to do with considering a module for inclusion in the Python stdlib. Besides, I don't understand how being packaged by a distro makes a difference. My distro has thousands of packages, many of them quite obscure. OTOH, being shipped in the stdlib *and* visibly documented on python.org (in the stdlib docs, in the what's new, etc.) will make a difference. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Sat, Jan 28, 2012 at 5:04 AM, Paul Moore wrote: > On 28 January 2012 09:18, Nick Coghlan wrote: > >> It's basically us saying to Python users "We're explicitly flagging >> this PyPI module for inclusion in the next major Python release. We've >> integrated it into our build process, test suite and binary releases, >> so you don't even have to download it from PyPI in order to try it >> out, you can just import it from the __preview__ namespace (although >> you're still free to download it from PyPI if you prefer - in fact, if >> you need to support multiple Python versions, we actively recommend >> it!). There's still a small chance this module won't make the grade >> and will be dropped from the standard library entirely (that's why >> it's only a preview), but most likely it will move into the main part >> of the standard library with full backwards compatibility guarantees >> in the next release". > > +1. Hm. You could do this just as well without a __preview__ package. You just flag the module as experimental in the docs and get on with your life. We have some experience with this in Google App Engine. We used to use a separate "labs" package in our namespace and when packages were deemed stable enough they were moved from labs to non-labs. But the move always turned out to be a major pain, causing more breakage than we would have had if we had simply kept the package location the same but let the API mutate. Now we just put new, experimental packages in the right place from the start, and put a loud "experimental" banner on all pages of their docs, which is removed once the API is stable. There is much less pain now: while incompatible changes do happen for experimental package, they are not frequent, and rarely earth-shattering, and usually the final step is simply removing the banner without making any (incompatible) changes to the code. This means that the final step is painless for early adopters, thereby rewarding them for their patience instead of giving them one final kick while they sort out the import changes. So I do not support the __preview__ package. I think we're better off flagging experimental modules in the docs than in their name. For the specific case of the regex module, the best way to adoption may just be to include it in the stdlib as regex and keep it there. Any other solution will just cause too much anxiety. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Hashing proposal: 64-bit hash
27.01.12 23:08, Frank Sievertsen написав(ла): As already mentioned, the vulnerability of 64-bit Python rather theoretical and not practical. The size of the hash makes the attack is extremely unlikely. Unfortunately this assumption is not correct. It works very good with 64bit-hashing. It's much harder to create (efficiently) 64-bit hash-collisions. But I managed to do so and created strings with a length of 16 (6-bit)-characters (a-z, A-Z, 0-9, _, .). Even 14 characters would have been enough. You need less than twice as many characters for the same effect as in the 32bit-world. The point is not the length of the string, but the size of string space for inspection. To search for a string with a specified 64-bit hash to iterate over 2 ** 64 strings. Spending on a single string scan 1 nanosecond (a very optimistic estimate), it would take 2 ** 64 / 1e9 / (3600 * 24 * 365.25) = 585 years. For the attack we need to find 1000 such strings -- more than half a million years. For 32-bit hash would need only an hour. Of course, to calculate the hash function to use secure, not allowing "cut corners" and reduce computation time. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] plugging the hash attack
Am 28.01.2012 02:19, schrieb Benjamin Peterson: > Hello everyone, > In effort to get a fix out before Perl 6 goes mainstream, Barry and I > have decided to pronounce on what we want for our stable releases. > What we have decided is that > 1. Simple hash randomization is the way to go. We think this has the > best chance of actually fixing the problem while being fairly > straightforward such that we're comfortable putting it in a stable > release. > 2. It will be off by default in stable releases and enabled by an > envar at runtime. This will prevent code breakage from dictionary > order changing as well as people depending on the hash stability. FWIW, the same will be done for 3.2. Georg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Jan 28, 2012, at 09:15 AM, Guido van Rossum wrote: >So I do not support the __preview__ package. I think we're better off >flagging experimental modules in the docs than in their name. For the >specific case of the regex module, the best way to adoption may just >be to include it in the stdlib as regex and keep it there. Any other >solution will just cause too much anxiety. +1 What does the PEP give you above this "simple as possible" solution? -Barry ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Sat, 28 Jan 2012 13:14:36 -0500 Barry Warsaw wrote: > On Jan 28, 2012, at 09:15 AM, Guido van Rossum wrote: > > >So I do not support the __preview__ package. I think we're better off > >flagging experimental modules in the docs than in their name. For the > >specific case of the regex module, the best way to adoption may just > >be to include it in the stdlib as regex and keep it there. Any other > >solution will just cause too much anxiety. > > +1 > > What does the PEP give you above this "simple as possible" solution? "I think we'll just see folks using the unstable APIs and then complaining when we remove them, even though they *know* *upfront* that these APIs will go away." That problem would be much worse if some modules were simply marked "experimental" in the doc, rather than put in a separate namespace. You will see people copying recipes found on the internet without knowing that they rely on unstable APIs. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3 benchmarks
On Sat, 28 Jan 2012 14:21:18 +0200 Maciej Fijalkowski wrote: > Hi > > Something that's maybe worth mentioning is that the "official" python > benchmark suite http://hg.python.org/benchmarks/ has a pretty > incomplete set of benchmarks for python 3 compared to say what we run > for pypy: https://bitbucket.org/pypy/benchmarks I think a very > worthwhile project would be to try to port other benchmarks (that > actually use existing python projects like sympy or django) for those > that has been ported to python 3. Agreed. cheers Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
Antoine Pitrou wrote: >On Sat, 28 Jan 2012 13:14:36 -0500 >Barry Warsaw wrote: >> On Jan 28, 2012, at 09:15 AM, Guido van Rossum wrote: >> >> >So I do not support the __preview__ package. I think we're better >off >> >flagging experimental modules in the docs than in their name. For >the >> >specific case of the regex module, the best way to adoption may just >> >be to include it in the stdlib as regex and keep it there. Any other >> >solution will just cause too much anxiety. >> >> +1 >> >> What does the PEP give you above this "simple as possible" solution? > >"I think we'll just see folks using the unstable APIs and then >complaining when we remove them, even though they *know* *upfront* that >these APIs will go away." > >That problem would be much worse if some modules were simply marked >"experimental" in the doc, rather than put in a separate namespace. >You will see people copying recipes found on the internet without >knowing that they rely on unstable APIs. How. About doing them the way we do depreciated modules, and have them spit warnings to stderr? Maybe add a flag and environment variable to disable that. -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
Le samedi 28 janvier 2012 à 10:46 -0800, Mike Meyer a écrit : > Antoine Pitrou wrote: > > >On Sat, 28 Jan 2012 13:14:36 -0500 > >Barry Warsaw wrote: > >> On Jan 28, 2012, at 09:15 AM, Guido van Rossum wrote: > >> > >> >So I do not support the __preview__ package. I think we're better > >off > >> >flagging experimental modules in the docs than in their name. For > >the > >> >specific case of the regex module, the best way to adoption may just > >> >be to include it in the stdlib as regex and keep it there. Any other > >> >solution will just cause too much anxiety. > >> > >> +1 > >> > >> What does the PEP give you above this "simple as possible" solution? > > > >"I think we'll just see folks using the unstable APIs and then > >complaining when we remove them, even though they *know* *upfront* that > >these APIs will go away." > > > >That problem would be much worse if some modules were simply marked > >"experimental" in the doc, rather than put in a separate namespace. > >You will see people copying recipes found on the internet without > >knowing that they rely on unstable APIs. > > How. About doing them the way we do depreciated modules, and have them > spit warnings to stderr? Maybe add a flag and environment variable to > disable that. You're proposing that new experimental modules spit warnings when you use them? I don't think that's a good way of promoting their use :) (something we do want to do even though we also want to convey the idea that they're not yet "stable" or "fully approved") Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
Michael Foord wrote: On 28/01/2012 04:44, Stephen J. Turnbull wrote: I think it's a bad idea to introduce a feature that's *supposed* to break (in the sense of "make a break", ie, change the normal pattern) with every release and then try to avoid breaking (in the sense of "causing an unexpected failure") code written by people who don't want to follow the discipline of keeping up with changing APIs. If they want that stability, they should wait for the stable release. Modules should become unavailable from __preview__ as soon as they have a stable home. I like not breaking people's code where *possible*. __preview__ is not about stability. It's about making code easily available for testing before the API freezes. If nothing has changed once it graduates, how hard is it to change a few lines of code from from __preview__ import blahblahblah to import blahblahblah ? It seems to me that including a __preview__ package in production software is a mistake, and not its intention. ~Ethan~ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP for allowing 'raise NewException from None'
Nick Coghlan wrote: On Sat, Jan 28, 2012 at 10:33 AM, Ethan Furman wrote: So the question is: - should 'raise ... from ...' be legal outside a try block? - should 'raise ... from None' be legal outside a try block? Given that it would be quite a bit of work to make it illegal, my preference is to leave it alone. I believe that means there's only one open question. Should "raise ex from None" be syntactic sugar for: 1. clearing the current thread's exception state (as I believe Ethan's patch currently does), thus meaning that __context__ and __cause__ both end up being None 2. setting __cause__ to None (so that __context__ still gets set normally, as it is now when __cause__ is set to a specific exception), and having __cause__ default to a *new* sentinel object that indicates "use __context__" I've already stated my own preference in favour of 2 - that approach means developers that think about it can explicitly change exception types such that the context isn't displayed by default, but application and framework developers remain free to insert their own exception handlers that *always* report the full exception stack. The reasoning behind choice two makes a lot of sense. My latest effort (I should be able to get the patch posted within two days) involves creating a new dummy exception, SuppressContext, and 'raise ... from None' sets cause to it; the printing logic checks to see if cause is SuppressContext, and if so, prints neither context nor cause. Not exactly how Nick describes it, but as far as I've gotten in my Python core hacking skills. ;) ~Ethan~ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] threading.Semaphore()'s counter can become negative for non-ints
Hello python-dev, This is probably worth of a bug report: While looking at threading.py I noticed that Semaphore's counter can go below zero. This is opposed to the docs: "The counter can never go below zero; ...". Just try: import threading s = threading.Semaphore(0.5) # You can now acquire s as many times as you want! # even when s._value < 0. The fix is tiny: diff -r 265d35e8fe82 Lib/threading.py --- a/Lib/threading.py Fri Jan 27 21:17:04 2012 + +++ b/Lib/threading.py Sat Jan 28 21:22:04 2012 +0200 @@ -322,7 +321,7 @@ rc = False endtime = None self._cond.acquire() -while self._value == 0: +while self._value <= 0: if not blocking: break if __debug__: Which is better than forcing s._value to be an int. I also think that the docs should be updated to reflect that the counter is not compared to be equal to zero, but non-positive. e.g. "when acquire() finds that it is zero...", "If it is zero on entry, block...". On another commit: Regarding http://bugs.python.org/issue9346, an unused import was left: -from collections import deque Cheers, TB ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] threading.Semaphore()'s counter can become negative for non-ints
2012/1/28 T.B. : > Hello python-dev, > > This is probably worth of a bug report: While looking at threading.py I > noticed that Semaphore's counter can go below zero. This is opposed to the > docs: "The counter can never go below zero; ...". Just try: > > import threading > s = threading.Semaphore(0.5) But why would you want to pass a float? It seems like API abuse to me. -- Regards, Benjamin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Hashing proposal: 64-bit hash
Zitat von Serhiy Storchaka : 27.01.12 23:08, Frank Sievertsen написав(ла): As already mentioned, the vulnerability of 64-bit Python rather theoretical and not practical. The size of the hash makes the attack is extremely unlikely. Unfortunately this assumption is not correct. It works very good with 64bit-hashing. It's much harder to create (efficiently) 64-bit hash-collisions. But I managed to do so and created strings with a length of 16 (6-bit)-characters (a-z, A-Z, 0-9, _, .). Even 14 characters would have been enough. You need less than twice as many characters for the same effect as in the 32bit-world. The point is not the length of the string, but the size of string space for inspection. To search for a string with a specified 64-bit hash to iterate over 2 ** 64 strings. I think you entirely missed the point of Frank's message. Despite your analysis that it shall not be possible, Frank has *actually* computed colliding strings, most likely also for a specified hash value. Of course, to calculate the hash function to use secure, not allowing "cut corners" and reduce computation time. This issue wouldn't be that relevant if there wasn't a documented algorithm to significantly reduce the number of tries you need to make to produce a string with a desired hash value. My own implementation would need 2**33 tries in the worst case (for a 64-bit hash value); thanks to the birthday paradox, it's actually a significant chance that the algorithm finds collisions even faster. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
Antoine Pitrou wrote: >Le samedi 28 janvier 2012 à 10:46 -0800, Mike Meyer a écrit : >> Antoine Pitrou wrote: >> >You will see people copying recipes found on the internet without >> >knowing that they rely on unstable APIs. >> >> How. About doing them the way we do depreciated modules, and have >them >> spit warnings to stderr? Maybe add a flag and environment variable >to >> disable that. > >You're proposing that new experimental modules spit warnings when you >use them? To be explicit, when the system loada them. > I don't think that's a good way of promoting their use :) And importing something from __preview__or __experimental__or whatever won't? This thread did include the suggestion that they go into their final location instead of a magic module. >(something we do want to do even though we also want to convey the idea >that they're not yet "stable" or "fully approved") Doing it with a message pointing at the page describing the status makes sure users read the docs before using them. That solves the problem of using them without realizing it. -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
> >You're proposing that new experimental modules spit warnings when you > >use them? > > To be explicit, when the system loada them. There are many reasons to import a module, such as viewing its documentation. And the warning will trigger if the import happens in non-user code, such as a library; or when there is a fallback for the module not being present. People usually get annoyed by intempestive warnings which don't warn about an actual problem. > >(something we do want to do even though we also want to convey the idea > >that they're not yet "stable" or "fully approved") > > Doing it with a message pointing at the page describing the status > makes sure users read the docs before using them. Sure, it's just much less user-friendly than conveying that idea in the module's namespace. Besides, it only works if warnings are not silenced. People are used to __future__ (and I've seen no indication that they don't like it). __preview__ is another application of the same pattern (using a special namespace to indicate the status of a feature). Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Sat, Jan 28, 2012 at 3:02 PM, Antoine Pitrou wrote: > There are many reasons to import a module, such as viewing its > documentation. And the warning will trigger if the import happens in > non-user code, such as a library; or when there is a fallback for the > module not being present. People usually get annoyed by intempestive > warnings which don't warn about an actual problem. As an alternative, how about a __preview__ or __provisional__ attribute on modules that are in this provisional state? So just add that big warning to the docs, as Guido suggested, and set the attribute as a programmatic indicator. Perhaps also add sys.provisional_modules (or wherever) to explicitly give the full list for the current Python version. -eric ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
Le samedi 28 janvier 2012 à 16:03 -0700, Eric Snow a écrit : > On Sat, Jan 28, 2012 at 3:02 PM, Antoine Pitrou wrote: > > There are many reasons to import a module, such as viewing its > > documentation. And the warning will trigger if the import happens in > > non-user code, such as a library; or when there is a fallback for the > > module not being present. People usually get annoyed by intempestive > > warnings which don't warn about an actual problem. > > As an alternative, how about a __preview__ or __provisional__ > attribute on modules that are in this provisional state? So just add > that big warning to the docs, as Guido suggested, and set the > attribute as a programmatic indicator. Perhaps also add > sys.provisional_modules (or wherever) to explicitly give the full list > for the current Python version. Well, how often do you examine the attributes of a module before using it? I think that's a much too obscure way to convey the information. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Sat, Jan 28, 2012 at 4:08 PM, Antoine Pitrou wrote: > Le samedi 28 janvier 2012 à 16:03 -0700, Eric Snow a écrit : >> On Sat, Jan 28, 2012 at 3:02 PM, Antoine Pitrou wrote: >> > There are many reasons to import a module, such as viewing its >> > documentation. And the warning will trigger if the import happens in >> > non-user code, such as a library; or when there is a fallback for the >> > module not being present. People usually get annoyed by intempestive >> > warnings which don't warn about an actual problem. >> >> As an alternative, how about a __preview__ or __provisional__ >> attribute on modules that are in this provisional state? So just add >> that big warning to the docs, as Guido suggested, and set the >> attribute as a programmatic indicator. Perhaps also add >> sys.provisional_modules (or wherever) to explicitly give the full list >> for the current Python version. > > Well, how often do you examine the attributes of a module before using > it? I think that's a much too obscure way to convey the information. Granted. However, actively looking for the attribute is only one of the lesser use-cases. The key is that it allows you to check any library programmatically for dependence on any of the provisional modules. The warning in the docs is important, but being able to have code check for it is important too. As a small bonus, it would show up in help for the module and in dir(). -eric ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Hashing proposal: 64-bit hash
The point is not the length of the string, but the size of string space for inspection. To search for a string with a specified 64-bit hash to iterate over 2 ** 64 strings. Spending on a single string scan 1 nanosecond (a very optimistic estimate), it would take 2 ** 64 / 1e9 / (3600 * 24 * 365.25) = 585 years. For the attack we need to find 1000 such strings -- more than half a million years. For 32-bit hash would need only an hour. With meet-in-the-middle and some other tricks it's possible to generate 25,000 64-bit-collisions per hour using an older desktop-cpu and 4gb ram. H_dypmRNWgOxiaaG A_ceO8B4Q2eKfabi S_kpgdB3tUFJiaae H_dypmRNWgOxiaaG D_FYzdys3H8qbaba 0_pOwRq15h8vbabO S_kpgdB3tUFJiaae __mdKp1GvI_fcaaM 6_U3_B0pJT1UsaaW 4_1GnK9BmLj9naa5 __X7hMeAOpACdaaw B_7pm.T62SiLlaai I_HSdl0axd8tmaae T_Dv3LwayACpdaaO Frank ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On 1/28/2012 3:55 AM, Nick Coghlan wrote: I am currently -something on the proposal as it because it will surely create a lot of hassles and because I do not think it is necessary the best solution to the motivating concerns. Don't consider this PEP a purely theoretical proposal, because it isn't. It's really being put forward to solve a specific problem: the fact that we need to do something about re's lack of proper Unicode support [1]. Those issues are actually hard to solve, so replacing re with Matthew Barnett's regex module (just as re itself was a replacement for the original regex module) that already addresses most of them seems like a good way forward, but this is currently being blocked because there are still a few lingering concerns with maintainability and backwards compatibility. I find the concern about 'maintainability' a bit strange as regex seems to be getting more maintainance and improvement than re. The re author is no longer active. If neither were in the library, and we were considering both, regex would certainly win, at least from a user view. Tom Christiansen reviewed about 8 unicode-capable extended r. e. packages, including both re and regex, and regex came out much better. The concern about back compatibility ignores the code that re users cannot write. In any case, that problem would be solved by adding regex in addition to re instead of as a replacement. If it were initially added as __preview__.regex, would the next step be to call it regex? or change it to re and remove the current package?. If the former, I think we might as well do it now. If the latter, that is different from what the pep proposes. While regex is the current poster-child for this problem, I see it as a special case that is not really addressed by the Pep. The other proposed use-case for __preview__ is packages whose api is not stable. Such packages may need their api changed a lot sooner than 18-24 months. Or, their api may change for a lot longer than just one release cycle. So the PEP would be best suited for packages who api may be fixed but might need code-breaking adjustments *once* in 18 months. A counter-proposal: add an __x__ package to site-packages. Document the contents separately in an X-Library manual. Let the api of such packages change with every micro release. Don't guarantee that modules won't disappear completely. Don't put a time limit on residence there before being moved up (to the stdlib) or out. Packages that track volatile external standards could stay there indefinitely. If an module is moved to stdlib, leave a stub for at least two versions that emits a deprecation warning (to switch to import a instead of __x__.a) and a notice that the doc has moved, along with importing the contents of the stdlib version. (This would work for the __preview__ proposal also.) -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Sun, Jan 29, 2012 at 3:15 AM, Guido van Rossum wrote: > Hm. You could do this just as well without a __preview__ package. You > just flag the module as experimental in the docs and get on with your > life. > > We have some experience with this in Google App Engine. We used to use > a separate "labs" package in our namespace and when packages were > deemed stable enough they were moved from labs to non-labs. But the > move always turned out to be a major pain, causing more breakage than > we would have had if we had simply kept the package location the same > but let the API mutate. Now we just put new, experimental packages in > the right place from the start, and put a loud "experimental" banner > on all pages of their docs, which is removed once the API is stable. > > There is much less pain now: while incompatible changes do happen for > experimental package, they are not frequent, and rarely > earth-shattering, and usually the final step is simply removing the > banner without making any (incompatible) changes to the code. This > means that the final step is painless for early adopters, thereby > rewarding them for their patience instead of giving them one final > kick while they sort out the import changes. > > So I do not support the __preview__ package. I think we're better off > flagging experimental modules in the docs than in their name. For the > specific case of the regex module, the best way to adoption may just > be to include it in the stdlib as regex and keep it there. Any other > solution will just cause too much anxiety. I'm willing to go along with that (especially given your report of AppEngine's experience with the "labs" namespace). Can we class this as a pronouncement on PEP 408? That is, "No to adding a __preview__ namespace, but yes to adding regex directly for 3.3"? Regards, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Sat, Jan 28, 2012 at 5:33 PM, Nick Coghlan wrote: > I'm willing to go along with that (especially given your report of > AppEngine's experience with the "labs" namespace). > > Can we class this as a pronouncement on PEP 408? That is, "No to > adding a __preview__ namespace, but yes to adding regex directly for > 3.3"? Yup. We seem to have a tendency to over-analyze decisions a bit lately (witness the hand-wringing about the hash collision DoS attack). For those who worry about people who copy recipes that stop working, I think they're worrying too much. If people want to take a shortcut without reading the documentation or understanding the code they are copying, fine, but they should realize the limitations of free advice. I don't mean to put down the many great recipes that exist or the value of copying code to get started quickly. But I think our liability as maintainers of the library is sufficiently delineated when we clearly mark a module as experimental in the documentation. (Recipe authors should ideally also add this warning to their recipe if it depends on an experimental API.) Finally, if you really want to put warnings in whenever an experimental module is being used, make it a silent warning, like SilentDeprecationWarning. That allows people to request more strict warnings without unduly alarming the users of an app. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 408 -- Standard library __preview__ package
On Sun, Jan 29, 2012 at 1:29 PM, Guido van Rossum wrote: > On Sat, Jan 28, 2012 at 5:33 PM, Nick Coghlan wrote: >> I'm willing to go along with that (especially given your report of >> AppEngine's experience with the "labs" namespace). >> >> Can we class this as a pronouncement on PEP 408? That is, "No to >> adding a __preview__ namespace, but yes to adding regex directly for >> 3.3"? > > Yup. We seem to have a tendency to over-analyze decisions a bit lately > (witness the hand-wringing about the hash collision DoS attack). I have now updated PEP 408 accordingly (i.e. rejected, but with a specific note about regex). And (since Alex Gaynor brought it up off-list), I'll explicitly note here that I'm taking your approval as granting the special permission PEP 399 needs to accept a C extension module without a pure Python equivalent. Patches to *add* a pure Python version for use by other implementations are of course welcome (in practice, I suspect it's likely only in PyPy that such an engine would be fast enough to be usable). Regards, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
