Re: [Python-Dev] Lazy unpacking for struct module
Le dimanche 12 juin 2011 à 10:27 -0700, Raymond Hettinger a écrit : > I do suggest that you publish your code as a third-party module > to make the optional available and to validate whether there > is any real interest in this. See the Hachoir project: it is a lazy parser supporting sub-structures (whereas struct only supports flat structures). It is difficult to implement a lazy parser: I chose to use Python generators to implement them. Hachoir should not enter Python standard library: it evoles too fast and it is too big (60K lines of Python). See also: bdec: http://www.protocollogic.com/ Construct: http://construct.wikispaces.com/ FileAlyzer: http://www.safer-networking.org/fr/filealyzer/index.html DataWorkshop: http://www.dataworkshop.de/ DataWorkshop project is dead since 2005. I don't remember if FileAlyzer is a free software or not. I agree with Raymond: struct module should be kept simple, and if you want a lazy parser: it should be a third party project. Victor ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] In-Python virtualisation and packaging
Back in March this year, Carl Meyer did some work to see if it was feasible to bring virtualenv functionality into Python [1] (code at [2]). Carl's changes were to Python code only, which was almost but not quite enough. I built on his changes with updates to C code in getpath.c/getpathp.c, and my code is at [3]. I've kept it synchronised with the core cpython repo, including the recently committed packaging modules. While there are issues to work through such as dealing with source builds (and no doubt plenty of others), it now seems possible to create virtual environments and install stuff into them using just the stdlib (modulo currently needing Distribute for the packages which don't yet support setup.cfg-based packaging, but it's all done automatically for a user). So you can do e.g. $ python3.3 -m virtualize /tmp/venv $ source /tmp/venv/bin/activate.sh $ pysetup3 install Mako and so on. A log of early experiments, which seems reasonably promising, is at [4]. Do people agree that it may be fitting, proper and timely to bring virtualisation into Python, and are there any fundamental flaws anyone can see with the approach used? If people want to experiment with this code without cloning and building, I created a Debian package using checkinstall, which can be installed using sudo dpkg -i pythonv_3.3-1_i386.deb and removed using sudo dpkg -r pythonv I can make this Debian package available for download, if anyone wants it. Regards, Vinay Sajip [1] http://mail.python.org/pipermail/distutils-sig/2011-March/017519.html [2] https://bitbucket.org/carljm/cpythonv [3] https://bitbucket.org/vinay.sajip/pythonv [4] https://gist.github.com/1022601 ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In-Python virtualisation and packaging
On 13/06/2011 12:47, Vinay Sajip wrote: Back in March this year, Carl Meyer did some work to see if it was feasible to bring virtualenv functionality into Python [1] (code at [2]). Carl's changes were to Python code only, which was almost but not quite enough. I built on his changes with updates to C code in getpath.c/getpathp.c, and my code is at [3]. I've kept it synchronised with the core cpython repo, including the recently committed packaging modules. While there are issues to work through such as dealing with source builds (and no doubt plenty of others), it now seems possible to create virtual environments and install stuff into them using just the stdlib (modulo currently needing Distribute for the packages which don't yet support setup.cfg-based packaging, but it's all done automatically for a user). So you can do e.g. $ python3.3 -m virtualize /tmp/venv $ source /tmp/venv/bin/activate.sh $ pysetup3 install Mako and so on. A log of early experiments, which seems reasonably promising, is at [4]. Do people agree that it may be fitting, proper and timely to bring virtualisation into Python, and are there any fundamental flaws anyone can see with the approach used? It would certainly need a PEP. There are two options: Bring the full functionality into the standard library so that Python supports virtual environments out of the box. As is the case with adding anything to the standard library it will then be impossible to add features to the virtualization support in Python 3.3 once 3.3 is released - new features will go into 3.4. Add only the minimal changes required to support a third-party virtual environment tool. Virtual environments are phenomenally useful, so I would support having the full tool in the standard library, but it does raise maintenance and development issues. Don't forget windows support! ;-) All the best, Michael Foord If people want to experiment with this code without cloning and building, I created a Debian package using checkinstall, which can be installed using sudo dpkg -i pythonv_3.3-1_i386.deb and removed using sudo dpkg -r pythonv I can make this Debian package available for download, if anyone wants it. Regards, Vinay Sajip [1] http://mail.python.org/pipermail/distutils-sig/2011-March/017519.html [2] https://bitbucket.org/carljm/cpythonv [3] https://bitbucket.org/vinay.sajip/pythonv [4] https://gist.github.com/1022601 ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In-Python virtualisation and packaging
On Mon, Jun 13, 2011 at 9:55 PM, Michael Foord wrote: > Virtual environments are phenomenally useful, so I would support having the > full tool in the standard library, but it does raise maintenance and > development issues. > > Don't forget windows support! ;-) Given that it is desirable for tools like virtualenv to support *old* versions of Python on *new* versions of operating systems, it seems to me that there is an inherent element of their feature set that makes including the whole tool questionable. OTOH, it may make sense to have a baseline tool provided innately, but provide the appropriate third party hooks to allow alternative tools to evolve independently of the stdlib. How well does the regression test suite cope when run inside such a virtualised environment? Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In-Python virtualisation and packaging
Michael Foord voidspace.org.uk> writes: > It would certainly need a PEP. Of course. > There are two options: > > Bring the full functionality into the standard library so that Python > supports virtual environments out of the box. As is the case with adding > anything to the standard library it will then be impossible to add > features to the virtualization support in Python 3.3 once 3.3 is > released - new features will go into 3.4. > > Add only the minimal changes required to support a third-party virtual > environment tool. > Agreed. As I see it, the "minimal changes required" are everything in my fork except for "virtualize.py", which was actually written as an external module "pmv.py" - Poor Man's Virtualenv ;-) Having it as a minimal implementation in the stdlib accords with "batteries included", but even as it stands, virtualize.py does try to cater for customisation. Firstly, there's a virtualizer_factory callable which can be overridden for customisation. That's called to produce a virtualizer, whose virtualize method is called with the target directory. The virtualize() function in virtualize.py just does this set of steps. I can't claim to have thought of everything, but it's a simple API which could have any number of implementations, not just the default one in the Virtualizer class in virtualize.py. > Don't forget windows support! I haven't. Though I haven't tested the most recent changes on Windows yet, I have tested the basic approach under Windows (the code doesn't rely on symlinks, but rather, copies of executables/DLLs). (All Windows testing so far has admittedly been using source builds rather than via a binary installer.) Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In-Python virtualisation and packaging
Nick Coghlan gmail.com> writes: > Given that it is desirable for tools like virtualenv to support *old* > versions of Python on *new* versions of operating systems, it seems to > me that there is an inherent element of their feature set that makes > including the whole tool questionable. You're right in terms of the current Python ecosystem and 3.x adoption, because of course this approach requires support from Python itself in terms of its site.py code. However, virtual environments have a utility beyond supporting older Pythons on newer OSes, since another common use case is having different library environments sandboxed from each other on different projects, even if all those projects are using Python 3.3+. The virtualenv module does an intricate bootstrapping dance which needs to accommodate each successive Python version, so there's maintenance overhead that way, too. Carl Meyer, being a pip and virtualenv maintainer, will probably have useful views on this. > OTOH, it may make sense to have a baseline tool provided innately, but > provide the appropriate third party hooks to allow alternative tools > to evolve independently of the stdlib. Yes - I'm thinking that what I've proposed is the baseline tool, and the question is about what the virtualisation API needs to look like to allow third-party tools to progress independently of the stdlib but in an interoperable way (a bit like packaging, I suppose). > How well does the regression test suite cope when run inside such a > virtualised environment? https://gist.github.com/1022705 325 tests OK. 5 tests failed: test_email test_importlib test_lib2to3 test_packaging test_sysconfig test_importlib might be broken because I accidentally committed some changes to marshal.c while working on #12291. test_packaging fails because of #12313. test_email fails for a similar reason - Makefile.pre.in is missing test_email in LIBSUBDIRS. test_sysconfig is probably failing because of changes I made, and I'm not sure of test_lib2to3. I will investigate! Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In-Python virtualisation and packaging
On Mon, 13 Jun 2011 11:47:33 + (UTC) Vinay Sajip wrote: > > $ python3.3 -m virtualize /tmp/venv > $ source /tmp/venv/bin/activate.sh > $ pysetup3 install Mako > > and so on. A log of early experiments, which seems reasonably promising, is at > [4]. > > Do people agree that it may be fitting, proper and timely to bring > virtualisation into Python, and are there any fundamental flaws anyone can see > with the approach used? This sounds really great, and definitely needs a PEP so that we can ask many questions :) As a side-note, I think calling it "virtualization" is a recipe for confusion. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In-Python virtualisation and packaging
On Mon, Jun 13, 2011 at 10:57 PM, Antoine Pitrou wrote: > As a side-note, I think calling it "virtualization" is a recipe for > confusion. Indeed, OS level virtualisation pretty much has a lock on that term. "virtual environments" skates close to it but manages to avoid it well enough to avoid confusion. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In-Python virtualisation and packaging
On Mon, Jun 13, 2011 at 10:50 PM, Vinay Sajip wrote: > You're right in terms of the current Python ecosystem and 3.x adoption, > because > of course this approach requires support from Python itself in terms of its > site.py code. However, virtual environments have a utility beyond supporting > older Pythons on newer OSes, since another common use case is having different > library environments sandboxed from each other on different projects, even if > all those projects are using Python 3.3+. Yeah, even if the innate one struggles on later OS releases that changed things in a backwards incompatible way, it will still be valuable on the OS versions that are around at the time that version of Python gets released. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In-Python virtualisation and packaging
On Mon, Jun 13, 2011 at 10:22 PM, Vinay Sajip wrote: > Michael Foord voidspace.org.uk> writes: >> Don't forget windows support! > > I haven't. Though I haven't tested the most recent changes on Windows yet, I > have tested the basic approach under Windows (the code doesn't rely on > symlinks, but rather, copies of executables/DLLs). (All Windows testing so far > has admittedly been using source builds rather than via a binary installer.) You should be able to use symlinks even on Windows these days (although granted they won't on portable media that uses a non-symlink friendly filesystem, regardless of OS). Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In-Python virtualisation and packaging
Nick Coghlan gmail.com> writes: > You should be able to use symlinks even on Windows these days > (although granted they won't on portable media that uses a non-symlink > friendly filesystem, regardless of OS). Plus I'm not sure Windows XP supports true symlinks - I think you have to make do with "junctions" a.k.a. "reparse points" which are more shambolic than symbolic ;-) I know symlinks are available on Vista, Windows Server 2008 and later, but XP is still very common. Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In-Python virtualisation and packaging
On Mon, Jun 13, 2011 at 08:42, Vinay Sajip wrote: > Nick Coghlan gmail.com> writes: > > > You should be able to use symlinks even on Windows these days > > (although granted they won't on portable media that uses a non-symlink > > friendly filesystem, regardless of OS). > > Plus I'm not sure Windows XP supports true symlinks - I think you have to > make > do with "junctions" a.k.a. "reparse points" which are more shambolic than > symbolic ;-) I know symlinks are available on Vista, Windows Server 2008 > and > later, but XP is still very common. I don't think we have any stdlib support for junctions, although we could certainly add it. In 3.2 we added symlink support for files and directories, which as you say is a Vista and beyond feature. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In-Python virtualisation and packaging
Nick Coghlan gmail.com> writes: > On Mon, Jun 13, 2011 at 10:57 PM, Antoine Pitrou pitrou.net> > wrote: > > As a side-note, I think calling it "virtualization" is a recipe for > > confusion. > > Indeed, OS level virtualisation pretty much has a lock on that term. > "virtual environments" skates close to it but manages to avoid it well > enough to avoid confusion. Or as they involving encapsulating paths and libaries, perhaps we could call them "capsules" ;-) though I think the term virtualenv is pretty entrenched now in the Python community. Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In-Python virtualisation and packaging
On Jun 13, 2011, at 11:47 AM, Vinay Sajip wrote: >Do people agree that it may be fitting, proper and timely to bring >virtualisation into Python, and are there any fundamental flaws anyone can see >with the approach used? Yes, absolutely. We'll hash out the details when the PEP is published, and bikeshed on all the terminology, but I really think this would be a very powerful addition to the standard library, so +1. Hopefully, the maintenance issues can be sorted out. Question: how hard would it be to backport the work you've done to Python 3.2? Obviously I'm not saying it should be ported to the official 3.2 branch, but if *someone* were interested in doing so, would it be possible? Sounds like you can almost get there with stdlib changes, but would require a few C changes too (I haven't looked at the diff yet). I'm just wondering if the same API could be made available to Python 3.2 as a third party module. It sounds like "almost, but not quite". >If people want to experiment with this code without cloning and building, I >created a Debian package using checkinstall, which can be installed using > >sudo dpkg -i pythonv_3.3-1_i386.deb > >and removed using > >sudo dpkg -r pythonv > >I can make this Debian package available for download, if anyone wants it. Is the Debian packaging branch available too? I'd be happy to throw that in my PPA for Ubuntu users to play with. Cheers, -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In-Python virtualisation and packaging
Barry Warsaw python.org> writes: > Question: how hard would it be to backport the work you've done to Python 3.2? > Obviously I'm not saying it should be ported to the official 3.2 branch, but > if *someone* were interested in doing so, would it be possible? Sounds like > you can almost get there with stdlib changes, but would require a few C > changes too (I haven't looked at the diff yet). I'm just wondering if the > same API could be made available to Python 3.2 as a third party module. It > sounds like "almost, but not quite". I think it's feasible - as far as I know, there's nothing 3.3 specific about the changes that were made, other than just happening to be against the default branch. AFAIK the getpath.c/getpathp.c changes will also work on 3.2, as all they do is look for a config file in a specific place and read a path from it if it's there. If it's not there, no biggie. If it's there, it sets up the sys.prefix/sys.exec_prefix values from that path. Possibly Carl's original Python changes would be easier to work from, since the sysconfig stuff has now changed quite a bit because of packaging coming in to cpython. For one thing, the _INSTALL_SCHEMES dict is replaced by reading that data from a config file. > Is the Debian packaging branch available too? I'd be happy to throw that in > my PPA for Ubuntu users to play with. My Debian-packaging-fu is not that good, I'm afraid, so there's no branch for the .deb, as such. I made the package by running make and then sudo checkinstall -D --fstrans=no which takes forever (God knows why - it's many many minutes at 100% CPU) but eventually comes up with the .deb. Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] is anyone using Misc/RPM?
On Sun, 2011-06-12 at 15:37 -0500, Benjamin Peterson wrote: > If no one is using it, I'd like to delete it. I also don't think we > should be in business of distributing distribution specific files. FWIW, Fedora and RHEL don't use this particular .spec file; we roll our own. I can't speak for all of the other RPM-using distributions, of course. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Implement Aspect-oriented programming
Jiawei Li gmail.com> writes: > For example, the logging module is not very useful right now, as it requires sprinkling small one-liners all over my code - not exactly ideal. > Why not take a page from aspect-oriented programming and allow for injection of code with point cuts? If you're only interested in logging method entry and exit - in which case, you're not really using logging to its full potential - then an AOP style approach may work for you. But the point of logging is to send messages to yourself (and others) from your code, and an AOP approach will not lend itself to intelligent, context-sensitive messages. Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In-Python virtualisation and packaging
On Jun 13, 2011, at 04:00 PM, Vinay Sajip wrote: >My Debian-packaging-fu is not that good, I'm afraid, so there's no branch for >the .deb, as such. I made the package by running make and then > >sudo checkinstall -D --fstrans=no > >which takes forever (God knows why - it's many many minutes at 100% CPU) but >eventually comes up with the .deb. Ah, no I don't think that'll be helpful. I can probably reuse the python3.3 packaging stuff to do a PPA. (It takes that long because it basically does a `make install`.) -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Participation Requested: Survey about Open-Source Software Development
Hi, Drs. Jeffrey Carver, Rosanna Guadagno, Debra McCallum, and Mr. Amiangshu Bosu, University of Alabama, and Dr. Lorin Hochstein, University of Southern California, are conducting a survey of open-source software developers. This survey seeks to understand how developers on distributed, virtual teams, like open-source projects, interact with each other to accomplish their tasks. You must be at least 19 years of age to complete the survey. The survey should take approximately 15 minutes to complete. If you are actively participating as a developer, please consider completing our survey. Here is the link to the survey: http://goo.gl/HQnux We apologize for inconvenience and if you receive multiple copies of this email. This survey has been approved by The University of Alabama IRB board. Thanks, Dr. Jeffrey Carver Assistant Professor University of Alabama (v) 205-348-9829 (f) 205-348-0219 http://www.cs.ua.edu/~carver ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In-Python virtualisation and packaging
Barry Warsaw python.org> writes: > Ah, no I don't think that'll be helpful. I can probably reuse the python3.3 > packaging stuff to do a PPA. Okay, go for it! Is there a specific tutorial somewhere about making a PPA for Python? (As opposed to more generalised tutorials - or would they be sufficient?) > (It takes that long because it basically does a > `make install`.) I realise that, as well as recording what it's doing, but that part seems to happen fairly quickly. Then it says "Copying files to the temporary directory..." and that part seems to take forever. The whole deb is under 25MB, what could be taking many minutes? Regards, Vinay ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] is anyone using Misc/RPM?
>> I disagree. We certainly include PCbuild/*.vcproj, and Tools/msi, >> which are also "distribution-specific". Likewise, we have plenty >> of OSX-specific files (including special-casing for specific releases >> thereof). > > I should qualify: We shouldn't distribute distribution-specific files > for which we don't provide binaries. Hmm. We have VS6, VS7, and VS8 project files, OS/2 makefiles, and configure.in has specifics for Solaris, even though we don't provide binaries for any of these. I don't think that's a useful principle to follow. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] is anyone using Misc/RPM?
Am 13.06.2011 18:21, schrieb David Malcolm: > On Sun, 2011-06-12 at 15:37 -0500, Benjamin Peterson wrote: >> If no one is using it, I'd like to delete it. I also don't think we >> should be in business of distributing distribution specific files. > > FWIW, Fedora and RHEL don't use this particular .spec file; we roll our > own. > > I can't speak for all of the other RPM-using distributions, of course. I doubt any of the distributions uses it, but that's not it's purpose, either. Instead, it is meant for people who want to roll their own RPM. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3101 implementation vs. documentation
On Sat, Jun 11, 2011 at 4:29 PM, Greg Ewing wrote:
> Ben Wolfson wrote:
>>
>> You can't have an internal replacement
>> field in this part of the replacement field, so '{' can always safely
>> be assumed to be Just a Brace and not the start of a replacement
>> field, regardless of whether it's doubled,
>
> I'm worried that the rules in this area are getting too
> complicated for a human to follow. If braces are allowed
> as plain data between square brackets and/or vice versa,
> it's going to be a confusing mess to read, and there will
> always be some doubt in the programmer's mind as to whether
> they have to be escaped somehow or not.
>
> I'm inclined to think that any such difficult cases should
> simply be disallowed. If the docs say an identifier is required
> someplace, the implementation should adhere strictly to that.
There are two cases with the braces: attribute selection and item
selection. The docs say that attributes should be identifiers, and
that the argument name should be an integer or an identifier, but that
the item selector can essentially be an arbitrary string as long as it
doesn't contain ']', which indicates its end. The docs as they stand
suggest that braces in item selectors should be treated as plain data:
"""
Format strings contain “replacement fields” surrounded by curly braces
{}. Anything that is not contained in braces is considered literal
text, which is copied unchanged to the output. If you need to include
a brace character in the literal text, it can be escaped by doubling:
{{ and }}.
"""
Since it mentions escaping only in the context of how to get a brace
in literal text rather than in a replacement field.
Current behavior is to perform escapes in the format spec part of a
replacement field, and that, I think, makes sense, since there can be
an internal replacement field there. However, it's still pretty simple
to tell whether braces need to be escaped or not: a brace in the
format spec does need to be escaped, a brace before the format spec
doesn't.
> It's not *that* hard to parse an indentifier properly, and
> IMO any use case that requires putting arbitrary characters
> into an item selector is abusing the format mechanism and
> should be redesigned to work some other way.
If by "item selector" you mean (using the names from the grammar in
the docs) the element_index, I don't see why this should be the case;
dictionaries can contain non-identified keys, after all. If you mean
the attribute_name (and arg_name) parts, then requiring an identifier
(or an integer for arg_name) makes a lot more sense.
I assume that Talin had some reason for stating otherwise in the PEP
(one of the few things that does get explicitly said about the
field_name part), but I'm kind of at a loss for why; you would need to
have a custom __getattribute__ to exploit it, and it would be a lot
less confusing just to use __getitem__.
--
Ben Wolfson
"Human kind has used its intelligence to vary the flavour of drinks,
which may be sweet, aromatic, fermented or spirit-based. ... Family
and social life also offer numerous other occasions to consume drinks
for pleasure." [Larousse, "Drink" entry]
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] is anyone using Misc/RPM?
On Mon, 13 Jun 2011 21:03:18 +0200 "Martin v. Löwis" wrote: > >> I disagree. We certainly include PCbuild/*.vcproj, and Tools/msi, > >> which are also "distribution-specific". Likewise, we have plenty > >> of OSX-specific files (including special-casing for specific releases > >> thereof). > > > > I should qualify: We shouldn't distribute distribution-specific files > > for which we don't provide binaries. > > Hmm. We have VS6, VS7, and VS8 project files, OS/2 makefiles, and > configure.in has specifics for Solaris, even though we don't provide > binaries for any of these. I don't think that's a useful principle > to follow. Well, if we want to nitpick, all these files are for compilation, no for distribution ;) Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Lazy unpacking for struct module
Raymond Hettinger wrote: How would you describe the creation of a lazy result that keeps a reference to the underlying buffer? I'd call it a view. There is plenty of precedence for this kind of object in Python -- I gave a few examples before. The distinguishing feature of ropes, as I understand the term, is that you get a lazy object when you *combine* other objects, e.g. concatenating strings. That's not being asked for here -- there is only taking apart going on, not putting together. It's also different from the oft-rejected idea of lazy string slicing, because extracting a field would give you a separate object, e.g. an int or string, not a reference to the original data object being unpacked. So I really can't see what harm it could do, except for maybe a tiny performance reduction in the case where you extract all the fields, or refer to extracted fields repeatedly. Even if that turned out to be the case, and was considered objectionable, it could still be useful to provide view-oriented unpacking as an option. -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.x and bytes
Thank you all for the responses. Rather than reply to each, I just made
one big summary. :)
Martin v. Löwis wrote:
> Ethan Furman wrote:
>> # constants
>>
>> EOH = b'\r'[0]
>> CHAR = b'C'[0]
>> DATE = b'D'[0]
>> FLOAT = b'F'[0]
>> INT = b'I'[0]
>> LOGICAL = b'L'[0]
>> MEMO = b'M'[0]
>> NUMBER = b'N'[0]
>>
>> This is not beautiful code.
>
> In this case, I think the intent would be better captured with
>
> def ASCII(c):
> return c.encode('ascii')
>
> EOH = ASCII('\r') # 0D
> CHAR= ASCII('C') # 43
> DATE= ASCII('D') # 44
> FLOAT = ASCII('F') # 46
> INT = ASCII('I') # 49
> LOGICAL = ASCII('L') # 4C
> MEMO= ASCII('M') # 4D
> NUMBER = ASCII('N') # 4E
>
> This expresses the intent that a) these are really byte values,
> not characters, and b) the specific choice of byte values was
> motivated by ASCII.
Definitely easier to read. If I go this route I'll probably use ord(),
though, since ascii and unicode are the same for the first 127 chars,
and there will be plenty of places to error out with a more appropriate
message if I get garbage. Since I really don't care what the actual
integer values are, I'll skip those comments, too.
Hagen Fürstenau wrote:
> You still have the alternative
>
> EOH = ord('\r')
> CHAR = ord('C')
> ...
>
> which looks fine to me.
Yes it does. I just dislike the (to me unnecessary) extra function
call. For those tuning in late to this thread, these are workarounds
for this not working:
field_type = header[11] # field_type is now an int, not a 1-byte bstr
if field_type == r'C': # r'C' is a 1-byte bstr, so this always fails
Greg Ewing wrote:
> Guido van Rossum wrote:
>>> On Thu, May 19, 2011 at 1:43 AM, Nick Coghlan wrote:
Proposals to address this include:
- introduce a "character" literal to allow c'a' as an alternative
to ord('a')
>>
>> -1; the result is not a *character* but an integer.
>
> Would you be happier if it were spelled i'a' instead?
That would work for me, although I would prefer a'a' (for ASCII). :)
Stephen J. Turnbull wrote:
> Put mascara on a pig, and you have a pig with mascara on, not Bette
> Davis. I don't necessarily think you're doing anybody a service by
> making the hack of using ASCII bytes as mnemonics more beautiful. I
> think Martin's version is as beautiful as this code should get.
I'll either use Martin's or Nick's. The point of beauty here is the
ease of readability. I think less readable is worse, and we shouldn't
have to have ugly, hard to read code nor inefficient code just because
we have to deal with byte streams that aren't unicode.
Nick Coghlan wrote:
> Agreed, but:
>
> EOH, CHAR, DATE, FLOAT, INT, LOGICAL, MEMO, NUMBER = b'\rCDFILMN'
>
> is a shorter way to write the same thing.
>
> Going two per line makes it easier to mentally map the characters:
>
> EOH, CHAR = b'\rC'
> DATE, FLOAT = b'DF'
> INT, LOGICAL = b'IL'
> MEMO, NUMBER = b'MN'
Wow. I didn't realize that could be done. That very nearly makes up
for not being able to do it one char at a time.
Thanks, Nick!
~Ethan~
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In-Python virtualisation and packaging
On 06/13/2011 06:55 AM, Michael Foord wrote: > There are two options: > > Bring the full functionality into the standard library so that Python > supports virtual environments out of the box. As is the case with adding > anything to the standard library it will then be impossible to add > features to the virtualization support in Python 3.3 once 3.3 is > released - new features will go into 3.4. I think it's not hard to provide enough hooks to allow third-party tools to extend the virtualenv-creation process, while still having enough code in the stdlib to allow actual creation of virtualenvs. Virtualenv already has very few features, and doesn't get very much by way of new feature requests -- all the UI sugar and significant shell integration goes into Doug Hellmann's virtualenvwrapper, and I wouldn't foresee that changing. IOW, I don't think the maintenance concerns outweigh the benefits of being able to create virtualenvs with an out-of-the-box Python. > Add only the minimal changes required to support a third-party virtual > environment tool. > > Virtual environments are phenomenally useful, so I would support having > the full tool in the standard library, but it does raise maintenance and > development issues. > > Don't forget windows support! ;-) > > All the best, > > Michael Foord ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In-Python virtualisation and packaging
On 06/13/2011 08:07 AM, Nick Coghlan wrote: > On Mon, Jun 13, 2011 at 10:50 PM, Vinay Sajip wrote: >> You're right in terms of the current Python ecosystem and 3.x adoption, >> because >> of course this approach requires support from Python itself in terms of its >> site.py code. However, virtual environments have a utility beyond supporting >> older Pythons on newer OSes, since another common use case is having >> different >> library environments sandboxed from each other on different projects, even if >> all those projects are using Python 3.3+. > > Yeah, even if the innate one struggles on later OS releases that > changed things in a backwards incompatible way, it will still be > valuable on the OS versions that are around at the time that version > of Python gets released. FWIW, historically pretty much every new Python version has broken virtualenv, and new OS versions almost never have. Virtualenv isn't especially OS-dependent (not nearly as much as some other stdlib modules): the most OS-dependent piece is "shell activation", and that's a feature I would prefer to entirely leave out of the stdlib virtualenv (it's a convenience, not a necessity for virtualenv use, and the need to maintain it for a variety of OS shells is a maintenance burden I don't think Python should adopt). In fact, the only new-OS-version adjustment I can recall virtualenv needing to make is when Debian introduced dist-packages -- but even that doesn't really apply, as that was distro-packager change to Python itself. With a built-in virtualenv it would be the distro packagers responsibility to make sure their patch to Python doesn't break the virtualenv module. So I don't think a virtualenv stdlib module would be at all likely to break on a new OS release, if Python itself is not broken by that OS release. (It certainly wouldn't be the stdlib module most likely to be broken by OS changes, in comparison to e.g. shutil, threading...) Carl ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.x and bytes
At 03:11 PM 6/13/2011 -0700, Ethan Furman wrote: Nick Coghlan wrote: > Agreed, but: > > EOH, CHAR, DATE, FLOAT, INT, LOGICAL, MEMO, NUMBER = b'\rCDFILMN' > > is a shorter way to write the same thing. > > Going two per line makes it easier to mentally map the characters: > > EOH, CHAR = b'\rC' > DATE, FLOAT = b'DF' > INT, LOGICAL = b'IL' > MEMO, NUMBER = b'MN' Wow. I didn't realize that could be done. That very nearly makes up for not being able to do it one char at a time. You can still do it one at a time: CHAR, = b'C' INT, = b'I' ... etc. I just tried it with Python 3.1 and it works there. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In-Python virtualisation and packaging
On 06/13/2011 06:46 PM, Carl Meyer wrote: > FWIW, historically pretty much every new Python version has broken > virtualenv I should clarify that this is because of the delicate stdlib bootstrapping virtualenv currently has to do; the built-in virtualenv eliminates this entirely and will require much less maintenance for new Python versions. Carl ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In-Python virtualisation and packaging
On 14/06/2011 00:46, Carl Meyer wrote: [snip...] So I don't think a virtualenv stdlib module would be at all likely to break on a new OS release, if Python itself is not broken by that OS release. (It certainly wouldn't be the stdlib module most likely to be broken by OS changes, in comparison to e.g. shutil, threading...) And if we gain Carl as a Python committer to help maintain it, then I'd say it is worth doing for that reason alone... Michael Carl ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.x and bytes
On Tue, Jun 14, 2011 at 9:40 AM, P.J. Eby wrote: > You can still do it one at a time: > > CHAR, = b'C' > INT, = b'I' > ... > > etc. I just tried it with Python 3.1 and it works there. I almost mentioned that, although it does violate one of the "unwritten rules of the Zen" (in this case, "syntax shall not look like grit on Tim's monitor") Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3101 implementation vs. documentation
Ben Wolfson wrote: If by "item selector" you mean (using the names from the grammar in the docs) the element_index, I don't see why this should be the case; dictionaries can contain non-identified keys, after all. Of course they can, but that's not the point. The point is that putting arbitrary strings between [...] in a format spec without any form of quoting or requirement for bracket matching leads to something that's too confusing for humans to read. IMO the spec should be designed so that the format string can be parsed using the same lexical analysis rules as Python code. That means anything that is meant to "hang together" as a single unit, such as an item selector, needs to look like a single Python token, e.g. an integer or identifier. I realise this is probably more restrictive than the PEP suggests, but I think it would be better that way all round. -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In-Python virtualisation and packaging
On 13Jun2011 13:47, Vinay Sajip wrote: | Nick Coghlan gmail.com> writes: | | > On Mon, Jun 13, 2011 at 10:57 PM, Antoine Pitrou pitrou.net> | > wrote: | > > As a side-note, I think calling it "virtualization" is a recipe for | > > confusion. | > | > Indeed, OS level virtualisation pretty much has a lock on that term. | > "virtual environments" skates close to it but manages to avoid it well | > enough to avoid confusion. | | Or as they involving encapsulating paths and libaries, perhaps we could call | them "capsules" ;-) though I think the term virtualenv is pretty entrenched now | in the Python community. "virtualenv" by all means - we all know what is meant. But "virtualisation" - I also am -1 on that. Indeed, when I started reading this thread my expectation was wrong for that very reason. Same issue with "capsules" (yes I know you weren't serious) - too generic a term, too vague. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ It looked good-natured, she thought; Still it had very long claws and a great many teeth, so she felt it ought to be treated with respect. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In-Python virtualisation and packaging
On 13Jun2011 17:31, Vinay Sajip wrote: | Barry Warsaw python.org> writes: | > Ah, no I don't think that'll be helpful. I can probably reuse the python3.3 | > packaging stuff to do a PPA. | | Okay, go for it! Is there a specific tutorial somewhere about making a PPA for | Python? (As opposed to more generalised tutorials - or would they be sufficient?) | | > (It takes that long because it basically does a | > `make install`.) | | I realise that, as well as recording what it's doing, but that part seems to | happen fairly quickly. | | Then it says "Copying files to the temporary directory..." and that part seems | to take forever. The whole deb is under 25MB, what could be taking many minutes? [ wild speculation ... ] How does it decide what to copy? If it is a "blind" make-me-a-package tool it may be scanning the whole OS install or something (expensive but linear) and maybe then doing some ghastly O(n^2) changed file comparison. Inefficient comparison stuff leaks into the real world all the time; the Linux kernel installs have a "hardlinks" program which is one of my pet hates for this very reason - it runs over the modules trees looking for identical module files to hard link and if you've got several kernels lying around it is unforgivably slow. Or maybe it loads the package install db into memory and does something expensive to see what's not accounted for. [ end speculation, but nothing useful now follows ... ] Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ "He deserves death!" "Deserves it! I daresay he does. And many die that deserve life. Is it in your power to give it to them? Then do not be so quick to deal out death in judgement, for even the very wise may not see all ends." - Gandalf, _The Lord of the Rings_ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3101 implementation vs. documentation
On Mon, Jun 13, 2011 at 5:36 PM, Greg Ewing wrote:
> Ben Wolfson wrote:
>
>> If by "item selector" you mean (using the names from the grammar in
>> the docs) the element_index, I don't see why this should be the case;
>> dictionaries can contain non-identified keys, after all.
>
> Of course they can, but that's not the point. The point is
> that putting arbitrary strings between [...] in a format
> spec without any form of quoting or requirement for bracket
> matching leads to something that's too confusing for humans
> to read.
But there is a requirement for bracket matching: the "[" that opens
the element_index is matched by the next "]". Arguably (as Terry Reedy
said) this is also a form of quoting, in which the square brackets are
the quotation operators. It seems no more confusing to me than
allowing arbitrary strings between in '"..."'; those quotation marks
aren't even oriented. (Admittedly, syntax highlighting helps there.)
Compared to this: "{0: ^+#10o}", a string like this: "this is normal
text, but {e.underline[this text is is udnerlined {sic}!]}---and we're
back to normal now" is pretty damn readable to this human, nor do I
see what about the rule "when you see a [, keep going until you see a
]" is supposed to be insuperably confusing. (Compare---not that it
helps my case in regard to readability---grouping in regular
expressions, where you don't usually have the aid of special syntax
highlighting inside the string; you see a '(', you know that you've
encountered a group which continues until the next (unescaped!) ')'.
The stuff that comes inside the parentheses might look like line
noise---and the whole thing might look like line noise---but *that*
rule about the structure of a regexp is pretty straightforward.)
> IMO the spec should be designed so that the format string
> can be parsed using the same lexical analysis rules as
> Python code. That means anything that is meant to "hang
> together" as a single unit, such as an item selector,
> needs to look like a single Python token, e.g. an integer
> or identifier.
If that's the rationale, why not change the spec so that instead of this:
"{0[spam]}"
You do this:
"{0['spam']}"
? Hangs together; single Python token. Bonus: it would make it
possible for this to work:
(a) "{0['12']}".format({'12': 4})
whereas currently this:
"{0[12]}".format(...)
passes the integer 12 to __getitem__, and (a) passes the string "'12'".
(Discovery: the "abuse" of the format mechanism I want to perpetrate
via element_index can also be perpetrated with a custom __format__
method:
>>> class foo:
...def __format__(self, a): return a * 2
...
>>> "hello {0::![} world".format(foo())
'hello :![:![ world'
. So any reform to make it impossible to use str.format creatively
will have to be fairly radical. I actually think that my intended
abuse is actually a perfectly reasonable use, but it would be
disallowed if only integers and identifiers can be in the
element_index field.)
--
Ben Wolfson
"Human kind has used its intelligence to vary the flavour of drinks,
which may be sweet, aromatic, fermented or spirit-based. ... Family
and social life also offer numerous other occasions to consume drinks
for pleasure." [Larousse, "Drink" entry]
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In-Python virtualisation and packaging
On Jun 14, 2011, at 01:00 AM, Michael Foord wrote: >On 14/06/2011 00:46, Carl Meyer wrote: >> [snip...] >> So I don't think a virtualenv stdlib module would be at all likely to >> break on a new OS release, if Python itself is not broken by that OS >> release. (It certainly wouldn't be the stdlib module most likely to be >> broken by OS changes, in comparison to e.g. shutil, threading...) >> > >And if we gain Carl as a Python committer to help maintain it, then I'd say >it is worth doing for that reason alone... +1 -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.x and bytes
On 12:35 am, [email protected] wrote: On Tue, Jun 14, 2011 at 9:40 AM, P.J. Eby wrote: You can still do it one at a time: CHAR, = b'C' INT, �= b'I' ... etc. �I just tried it with Python 3.1 and it works there. I almost mentioned that, although it does violate one of the "unwritten rules of the Zen" (in this case, "syntax shall not look like grit on Tim's monitor") [CHAR] = b'C' [INT] = b'I' ... Jean-Paul ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Call For Topics for Python Language Summit at EuroPython 2011
Howdy howdy. Michael Foord can't make it to EuroPython this year--and congratulations to him on his new baby!--so he's asked me to chair the Python Language Summit in his stead. Although I have some suggested topics from Michael, I don't have any burning issues of my own to add to the docket. That's where *you* come in. If you have a topic you'd like to suggest, please email it privately to me (larry at hastings dot org) along with an estimate of how long you think it'll take to discuss. Please have all suggestions to me no later than Friday June 17th. Also, please include the phrase "Python Language Summit" in the subject line, in case it gets caught in my spam filters. You don't have to be attending the summit to suggest a topic--I'll consider suggestions from anybody. Hope to see you at EuroPython, /larry/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
