Re: [python-committers] Thank you Larry Hastings!
Thank you, Larry! Cheers, Nick. -- https://mail.python.org/mailman/listinfo/python-list
Re: when wxPython update to python2.4?
alang_yl wrote: > i can't wait. > wxPython 2.5 already has a version which works with Python 2.4 (grab it from www.wxpython.org). For the wxPython 2.4 series, I understand Robin is planning a release which will both integrate the 2.4 series into the new wxPython versioning scheme, and also include a Python 2.4 compatible version. The timeframe for that isn't entirely clear, though. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Frameworks for "Non-Content Oriented Web Apps"
[EMAIL PROTECTED] wrote: But is there some sort of framework or something that is actually meant for such web apps,application that make heavy use of forms, have very high amount of user interaction etc. Hmm, PJE's PEAK might be worth having a look at: http://peak.telecommunity.com/ However, I'm not sure if that will provide enough of the 'web' side of things. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: The Industry choice
Paul Rubin wrote: I don't see that big a problem. The current Python sorting routine operates on instances of class "object" and calls the __cmp__ method to do comparisons. Every class of sortable objects either defines a __cmp__ method or inherits one from some superclass, and sort calls those methods. Static type declarations would not require writing any additional sorting routines. Python's list.sort doesn't check the *type* of the arguments at all. It only looks for the relevant comparison methods (__cmp__ or __lt__, as I recall). Sure, classes written in *Python* will ultimately inherit from either object or types.ClassType, but extension classes need not do any such thing. Yet list.sort works with them all, anyway. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: PEP 288 ponderings
Ian Bicking wrote: Using a one-element list is kind of annoying, because it isn't clear out of context that it's just a way of creating shared state. But it's okay, work right now, and provides the exact same functionality. Uh, isn't shared state what classes were invented for? Py> class mygen(object): ... def __init__(self, data): ... self.data = data ... def __iter__(self): ... while 1: ... print self.data ... yield None ... Py> g = mygen(0) Py> giter = iter(g) Py> giter.next() 0 Py> g.data = 1 Py> giter.next() 1 Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: screen clear question
Alan Gauld wrote: But the bottom line is that there is no builtin command because the mechanism is different on each platform. I'd have said it was because the inpreter is line-oriented rather than screen-oriented, but YMMV. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: arbitrary number of arguments in a function declaration
rbt wrote: How do I set up a function so that it can take an arbitrary number of arguments? For example, I have a bunch of expenses which may grow or shrink depending on the client's circumstance and a function that sums them up... hard coding them is tedious. How might I make this dynamic so that it can handle any amount of expenses? def tot_expenses(self, e0, e1, e2, e3): pass The Python Tutorial is a wonderful thing. . . Anyway, you can either set up your function to take a proper list, and then discover that the sum function already exists to add up the contents of a list: def tot_expenses(self, expenses): self.total_expenses = sum(expenses) Or, have the function take a variable number of arguments, and do the same thing: def tot_expenses(self, *args): self.total_expenses = sum(args) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Deferred expressions (was Re: Lambda as declarative idiom)
Steven Bethard wrote: Nick Coghlan: def-from syntax [4] (def f(a) + o(b) - o(c) from (a, b, c)) (def x * x from (x)) (def x from ()) (def x.bar(*a, **k) from (*a, **k)) ((def x(*a, **k) from ()) for x, a, k in funcs_and_args_list) After a bit more musing, this is definitely my preferred syntax. I'd call it a 'deferred expression' rather than an anonymous function, though. The name change may seem minor, but it changes the emphasis of the feature from how it's implemented to what it is useful for (i.e. deferring evaluation of an expression until call time). It's also conducive to a clean no-argument syntax - simply make the 'from' claus optional when the argument list is empty. 'def' and 'from' are already keywords, so there shouldn't be any compatibility problems. Michael Spencer's idea of using 'for' instead of 'from' was quite interesting, but the use of 'for' without a corresponding 'in' feels a bit misleading :) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Hlelp clean up clumpsy code
It's me wrote: Another newbie question. There must be a cleaner way to do this in Python: section of C looking Python code a = [[1,5,2], 8, 4] a_list = {} i = 0 for x in a: if isinstance(x, (int, long)): x = [x,] for w in [y for y in x]: i = i + 1 a_list[w] = i print a_list # The code prints what I want but it looks so "C-like". How can I make it more Python like? Firstly, calling your dictionary "a_list" is evil. . . Secondly, explaining what you want the code to do in English is handy when asking for help cleaning up code (since we then know which features are deliberate, and which are accidental implementation artificacts). If I'm reading the code correctly, you want to flatten a data structure which may contain either substructures or actual elements. A custom generator will do nicely: Py> def flatten(seq): ... for x in seq: ... if hasattr(x, "__iter__"): ... for y in flatten(x): ... yield y ... else: ... yield x ... Py> data = [[1,5,2],8,4] Py> val_to_pos = {} Py> for i, x in enumerate(flatten(data)): ... val_to_pos[x] = i + 1 ... Py> print val_to_pos {8: 4, 1: 1, 2: 3, 4: 5, 5: 2} Not any shorter, but this version works correctly for any leaf elements which don't supply __iter__ (e.g. strings), and internal elements which do (e.g. tuples) and the depth is limited only by the maximum level of recursion. Don't try to flatten a circular structure, though :) You may not even need to write the generator, since if you have Tkinter, that already supplies a near-equivalent function: Py> from Tkinter import _flatten as flatten Py> data = [[1,5,2],8,4] Py> val_to_pos = {} Py> for i, x in enumerate(flatten(data)): ... val_to_pos[x] = i + 1 ... Py> print val_to_pos {8: 4, 1: 1, 2: 3, 4: 5, 5: 2} It even works with strings as leaf elements: Py> data = [["abc","def",2],8,"xyz"] Py> flatten(data) ('abc', 'def', 2, 8, 'xyz') Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: why does UserDict.DictMixin use keys instead of __iter__?
Steven Bethard wrote: Sorry if this is a repost -- it didn't appear for me the first time. So I was looking at the Language Reference's discussion about emulating container types[1], and nowhere in it does it mention that .keys() is part of the container protocol. Because of this, I would assume that to use UserDict.DictMixin correctly, a class would only need to define __getitem__, __setitem__, __delitem__ and __iter__. So why does UserDict.DictMixin require keys() to be defined? Because it's a DictMixin, not a ContainerMixin? .keys() is definitely part of the standard dictionary interface, and not something the mixin can derive from the generic container methods. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: why does UserDict.DictMixin use keys instead of __iter__?
Steven Bethard wrote: Nick Coghlan wrote: .keys() is definitely part of the standard dictionary interface, and not something the mixin can derive from the generic container methods. Why is that? Isn't keys derivable as: def keys(self): return list(self) if __iter__ is defined? As you may have guessed, I completely forgot about __iter__. . . Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: python 3000 and removal of builtin callable
Nicolas Fleury wrote: Mirko Zeibig wrote: This is not an option for e.g. IDEs as some functions might actually do something when called ;-) and I like `callable` for introspection. Other ways would be to check for the `__call__` attribute or use several methods of the `inspect`-Module, both of which are not better than `callable` IMHO. I totally agree with you. The callable function could be moved to a module and be built-in, but it cannot really be removed. Calling a callable and know if an object is a callable without calling it is definitely not the same thing. For many of the builtins to be "removed" in Py3k, I believe the actual intent is to move them to a module (e.g. sys), with the removal being "remove from the builtins", not "remove from Python". Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: [Python-Dev] Let's get rid of unbound methods
Andrew Koenig wrote: duck typing? That's the Australian pronunciation of "duct taping". More Kiwi, I'm thinking ;) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Securing a future for anonymous functions in Python
Paul Rubin wrote: Nick Coghlan <[EMAIL PROTECTED]> writes: Do you consider generator expressions or list comprehensions deficient because they don't allow several statements in the body of the for loop? I don't see what it would mean to do otherwise. Exactly the same as a suite would in the innermost portion of the equivalent explicit generator (i.e. where the implicit "yield " currently ends up). If you could put a suite inside a function expression (i.e. replacing the implicit "return ") why not inside generator expressions as well? Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Deferred expressions (was Re: Lambda as declarative idiom)
Bengt Richter wrote: I like the fact that 'def' can serve as a mnemonic for 'defer' or 'deferred' ;-) Yeah, me too. I didn't actually notice that until after I'd thought of the phrase. OTOH, I like concise notation for expressions, and the def and from aren't really necessary if you can tag the first expression with some syntax marker. I suggested (: rather than (def since (: is illegal now and won't break anything. True, but I've always liked Python's preference for keywords over punctuation. And 'def' is only 2 characters longer than ':', too. That is an advantage of having it inside the outer parens, which my (:expr)(params) couldn't benefit from -- unless I just take your format and substitute ':' for both def and from: But where does that leave "Python is executable pseudocode"? Compare: (lambda (a, b, c) : f(a) + o(b) - o(c)) (: f(a) + o(b) - o(c) : (a, b, c)) (def f(a) + o(b) - o(c) from (a, b, c)) Which of the above most clearly expresses "defer a functional expression taking arguments a, b, and c"? For comparison, named syntax requires 3 lines, since Guido also wants to remove suite one-liners in Py3k: def f1(a, b, c): return f(a) + o(b) - o(c) f1 Not so bad for a single deferred expression (particularly a not-completely-trivial one like this), but fares significantly worse for simple zero-argument functions, or if you are just setting up a couple of deferred expressions in a function call. There's no more virtue in completely squandering vertical screen real estate than there is in being overly conservative with it. What was the point of all this again? Pretending lambda goes away without really killing it? ;-) Actually, it was started by the "please don't take our lamdbas!" comments regarding Python 3k. I suggested that one of the problems with lambda is that it is an expression but looks like a one-liner statement ("it's ugly as sin" may have been closer to my actual phrasing), that the syntax would be deservedly rejected if proposed as a new addition to the language, and a more productive response might be to consider nicer alternative syntax that Guido would consider retaining in Python 3k (since it would make sense to try out such a syntax in the Python 2.x series before committing to it for 3.0). Of course, after making that suggestion, my brain decided to start working on its *own* ideas for a more Pythonic syntax (trying to learn from generator expressions), which seemed to set off a few other people :) The name 'deferred expression' is intended to emphasise what I see as the most appropriate application for anonymous functions - using an entire named function statement to defer a simple expression is serious overkill. Recognising this use case was what switched me from "I won't miss lambdas" to "You know, I'd really regret it if they disappeared completely". Basically, I think lambda has a PR problem - its usefulness as a way to defer expression evaluation is obscured by its ugly syntax and its current explanation as "a neutered form of function definition, but at least you can use it in an expression". On backticks, they have a major problem in that many fonts make ` and ' virtually indistinguishable. A secondary problem is that printed text often renders an opening single quote like a backtick. Guido has already mentioned this in one of Python Regrets talks (in the context of the backtick repr syntax). So I don't expect any syntax involving backticks to even be looked at by the BDFL. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Python evolution: Unease
Carlos Ribeiro wrote: Couldn't a better document-writing interface be implemented? Such as: http://www.python.org/moin/Documentation Or AMK's annotatable docs: http://pydoc.amk.ca/frame.html Something like that? The docs are great, but it took me some time to find them out after searching inside Python docs. This is not a minor issue IMHO. Actually, this is a fair point - I work on isolated machines a lot (i.e. no net access), which means I can't get to any of these handy things. It would be nice if at least some of these things could be bundled with the CPython interpreter and linked to from the main documentation (rather than from the sidebar menu on the webpage). Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Python evolution: Unease
[Daniel Bowett] #- Contribute to where on Sourceforge??? Which domentation are #- we talking #- about in general? Speaking of docs. . . I think it would help a great deal if the python.org version-specific documentation pages used the standard documentation front page that actually includes the "About the Python Documentation" link that explains how to do this. Compare: http://www.python.org/doc/2.4/ To: http://www.python.org/dev/doc/devel/ Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there any way/where to subscribe for automated PEP status emails?
Thomas Heller wrote: You could probably subscribe to python-checkins, and filter it. Or read it via gmane. Hmm - can SF be used to setup a mailing list just for checkins to a single directory in the source tree? If so, that would seem to be an easy way to provide a python-pep-updates mailing list. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: navigating/changing directories
The script is executed in a process separate from your command shell, and hence has no effect on your shell's current directory. There are some things that batch files and shell scripts are still good for - manipulating the shell :) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: get the IP address of a host
J Berends wrote: Lee Harr wrote: Basically, it scrapes the output from ifconfig for the actual address assigned to the interface. Works perfectly on FreeBSD and Linux (given the correct configuration). Nice way, have to device something for windows than. Use the same approach, but scrape the output of ipconfig instead. Use subprocess to run the command in Python 2.4, or work something out with one of the popen variants for earlier versions. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Getting rid of "self."
BJörn Lindqvist wrote: So I'm asking here if someone knows a better way, maybe using decorators or metaclasses or other black magic? Wait for Python 3k when this will work: class c: def __init__(self): with self: .x = 1 .y = 2 .hi = "Hi there!" Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: 2 versions of python on 1 machine
flupke wrote: Peter Hansen wrote: The content of each batch file is like this: @echo off c:\python23\python.exe %1 %2 %3 %4 %5 %6 %7 %8 %9 More recent versions of Windows should allow you to use %* for "all the arguments to the batch file". Where did you find more info on PYTHONHOME and PYTHONPATH because the docs don't seem to contain a whole lot of info. I think PYTHONPATH is discussed in the tutorial. PYTHONHOME, I'm not so sure on (PYTHONPATH and PYTHONSTARTUP are the only two you're likely to care about, though - and I think the Tutorial covers both of them). Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: sorting on keys in a list of dicts
Jeff Shannon wrote: I suppose that your version has the virtue that, if the sortkey value is equal, items retain the order that they were in the original list, whereas my version will sort them into an essentially arbitrary order. Is there anything else that I'm missing here? Stability in sorting is a property not to be sneezed at - it means switching to sorting by a second key gives the effect of "sort by key 1, then by key 2", whereas that doesn't hold with an unstable sort algorithm. If you've ever used an application with an unstable sorting process and that only allows sorting a table on one column at a time, you'll appreciate the frustration that can cause :) Also, it's required to match the behaviour of the Python 2.4 version (which gets to take advantage of the stability of the builtin sort). Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Securing a future for anonymous functions in Python
Jacek Generowicz wrote: [*] Funnily enough, getting them to understand that "lambda x: fn(x)" is just a very silly way of writing "fn", can be quite a struggle at times ... but that's probably a consequence of the context in which lambda is introduced. If you genuinely taught them that, you may have done them a disservice: Py> def f(x): ...print x ... Py> f1 = f Py> f2 = lambda x: f(x) Py> f1("hi") hi Py> f2("hi") hi Py> def f(x): ... print x * 2 ... Py> f1("hi") hi Py> f2("hi") hihi Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Securing a future for anonymous functions in Python
Paul Rubin wrote: "Anna" <[EMAIL PROTECTED]> writes: Having taken some calculus (derivatives, limits, some integrals) but never even heard of lambda calculus, to me, lambda means absolutely NOTHING. Less than nothing. Lambda calculus is from mathematical logic, but more to the point "lambda" has been the term used in Lisp for this operation since time immemorial. I think that's part of the problem though - people familiar with lambda calculus and Lisp's lambdas want Python's lambdas to be equally capable, and they just plain *aren't*. If you have a complex function, the Pythonic way is to give it a meaningful name. Having a way to defer evaluation of a simple expression *is* quite handy, but 'lambda' is the wrong name for it - the parallels to lambda calculus and Lisp's lambda functions are likely to be misleading, rather than helpful. Add in the fact that there are many, many Python programmers with non-CS backgrounds, and the term 'lambda' sticks out like a sore thumb from amongst Python's other English-based keywords. 'def' is probably the second-most cryptic when you first encounter it, but it is a good mnemonic for "define a function", so it's still easy to parse. "Lambda is the term mathematicians use to refer to an anonymous function" is nowhere near as grokkable ;) For me, the alternative syntax discussion is based on 3 of the 4 mentioned reasons: 1. The syntax I don't like re-using colons as something other than suite delimiters - it breaks up the affected expression too much (particularly function calls). Code with dict literals inside function calls bugs me for the same reason (it's OK when the literal is separated out into an assignment statement for the dict). It's also too easy to write lambdas which look ambiguous, even though they technically aren't. Finally, Python has a reputation as "executable pseudocode". Lambda expressions don't read like any sort of psuedocode you're likely to see outside a maths department. 2. The limitation to a single expression I consider this no more of a problem than the restriction to a single expression in the main loop of a generator expression or a list comprehension. When those get too complicated, you switch to using a real for loop somewhere. Deferred expressions are no different - when the guts get too complicated, switch to a named function. 3. The word 'lambda' itself This _is_ one of my objections for the reasons stated above: for people unfamiliar with the term, they don't know what it is; for people familiar with the term, it isn't what they think it should be. Python already has a perfectly good keyword for functions, which has the additional virtue of being half the length of lambda (this matters, since this is a keyword that gets embedded in expressions - all the other keywords currently in that category are three letters or less: and, or, is, in, for) 4. People complaining about 2 Oh hell yes, this bugs me. And I think changing the syntax and calling them "deferred expressions" instead of "lambdas" would go a long way towards eliminating the griping. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Please Contribute Python Documentation!
Skip Montanaro wrote: Fred Drake releases new development versions of the docs frequently (typically once every month or two). I normally use the development docs instead of the regular ones for my day-to-day work. They are available here: http://www.python.org/dev/doc/devel/ It's useful to check to see the most current status of the docs. The only thing more current is the source in CVS. Although, when working this way, and Python's behaviour seems to contradict the documentation, don't forget to check the docs specific to your version of Python before submitting a bug report :) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Packaging, dependencies and rates of development
Bulba! wrote: When you have it ALL in the single distro, released from time to time, you can easily test it _together_. You don't get _temporal dependencies between various versions_. The released, stable distro has the same things on the same days, and either it all works together or it doesn't. Except that the difference between the distro maintainers' opinion of what is "stable enough" for inclusion, and the updated version of that module which has some stability problems in areas you don't use but fixes a critical bug in the areas you *do* use is always going to be a problem that has to be dealt with. That's one of the reasons numarray hasn't hit the standard library yet - the Python version release and uptake cycle is too slow for the pace of development they currently require. Similarly, this is also part of the reason IDLE went the way of IDLEFork before being merged back in 2.3 - the development outside the core tree let Kurt and everyone else involved get it up to speed more quickly. There's also the fact that monolithic approaches just plain don't scale - imagining that they can is exactly what leads to the version conflicts that bug you so much, since developers assume that they don't need to care about versioning issues, because the packagers will take care of them. In reality, all it does is move the version conflicts currently noticed by end users and make the packagers try to deal with them instead - and for some of the problems, side-by-side installation of multiple version is the only solution, and that's impossible in the general case without the modules in question providing some versioning mechanism. And so, once again, we're left with developers having to worry about dependency issues. For an example of a well-thought out approach to the versioning issue, take a look at the work that was done for the recent wxPython release: http://wiki.wxpython.org/index.cgi/MultiVersionInstalls What would be ideal is if distutils itself was able to offer a standard package versioning system. Then all module developers would have to do is be aware of how to use the distutils versioning system, and installing side-by-side versions of packages would be create straightforward. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: sorting on keys in a list of dicts
It's me wrote: What does it mean by "stability in sorting"? Can somebody please give a sample for using the code posted? I am a little lost here and I like to know more about the use of keys It's the jargon for what Jeff said - if you are sorting by some value calculated from each list entry, and different list entries may give the same value, then a stable sort guarantees that the order of entries giving the same value will be preserved from the original list. Consider: Py> from operator import itemgetter as item Py> seq = [(1, 1), (2, 1), (2, 3), (1, 5)] Py> seq.sort(key=item(1)) Py> seq #1 [(1, 1), (2, 1), (2, 3), (1, 5)] Py> seq.sort(reverse=True) Py> seq #2 [(2, 3), (2, 1), (1, 5), (1, 1)] Py> seq.sort(key=item(1)) Py> seq #3 [(2, 1), (1, 1), (2, 3), (1, 5)] This snippet sorts the tuples according to the second item, then sorts them in reverse order of the whole tuple, then resorts them according to the second item. Notice that the order of the first two items is different between point #1 and point #3. This is because sorting by the second item makes no distinction between these two tuples, and they retain whatever order they had before the sort began. This is what it means to have a stable sort, and it makes expressing complex sorting quite easy by chaining different sort operations together. Python 2.3 has a stable sort, and Python 2.4 brought the guarantee that it shall remain that way. I'm not sure about Python 2.2 and earlier. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Notification of PEP Updates
I can't recall which thread this came up in, so I'm starting a new one. . . Barry Warsaw has kindly added a "peps" topic to the python-checkins mailing list. If you want to be notified only when PEP's get updated, then subscribe to python-checkins and edit your settings to select just the 'peps' topic. Let us know if it does anything odd (e.g. sending updates about other checkins) The URL for the mailing list is: http://mail.python.org/mailman/listinfo/python-checkins I believe the mail you receive should contain the checkin messages, along with a summary of the differences between the old version and the new version (handy if you can read a context diff, not so handy otherwise). Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: sorting on keys in a list of dicts
Craig Ringer wrote: Well, that's several hundred more words than were probably required, but I hope I made sense. Remarkably similar to what I just posted. . . I guess numeric 2-tuples are just too good to pass up when discussing sorting :) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Securing a future for anonymous functions in Python
Jacek Generowicz wrote: Yes, I was wondering whether I should add lots of caveats to the above. I thought that might be the case. When teaching, I guess I can understand the need to avoid "well, technically, this isn't always true". It sounds like the students were having a tough enough time of it already :) Usually one or two have trouble grasping that "int" would be perfectly adequate in this situation. The ability to pass around functions at run-time was probably the hardest thing to get my head around when I started with Python, after a history of working in languages where "the compiler knows about types and functions, and the runtime knows about variables and instances". Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Developing Commercial Applications in Python
Stephen Waterbury wrote: A notable example is Verity's search engine -- see http://python.oreilly.com/news/PythonSS.pdf Not to mention the kind words of the current reigning king of the search engine world. . . Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Calling Function Without Parentheses!
Kamilche wrote: Uh, you're right! I wouldn't want to bog Python down with even more checking at run time. I guess I'm asking for syntax checks that are typically done only with compiled languages. In that case, I can suggest having a look at Pychecker, which performs static sanity checks on Python code. http://pychecker.sourceforge.net/ Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: What could 'f(this:that=other):' mean?
If the caller is meant to supply a namespace, get them to supply a namespace. def f(ns1, ns2): print ns1['a'], ns1['b'], ns2['a'], ns2['b'] f(ns1 = dict(a=1, b=2), ns2 = dict(a=3, b=4)) Hey, where's Steve? Maybe his generic objects should be called namespaces instead of bunches. . . def f(ns1, ns2): print ns1.a, ns1.b, ns2.a, ns2.b f(ns1 = namespace(a=1, b=2), ns2 = namespace(a=3, b=4)) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: sorting on keys in a list of dicts
Jeff Shannon wrote: Agreed. I'd started typing before I realized that it'd provide a stable sort, which pretty much answered my own question, but decided to send it anyhow in case I'd missed anything else... :) And it turns out we both missed the fact that it avoids comparing the dictionaries which could save a *lot* of number crunching (as well as making otherwise unsortable lists sortable). So it's a good thing you did decide to send it :) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Getting rid of "self."
Roy Smith wrote: It's actually kind of neat, but boy does it play headgames with me when I switch back and forth between that and Python. Switching back and forth betwen C++ and Python plays headgames *anyway* }:> Cheers, Nick. Hardware control with Python is nice. . . -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: python3: 'where' keyword
Andrey Tatarinov wrote: Hi. It would be great to be able to reverse usage/definition parts in haskell-way with "where" keyword. Since Python 3 would miss lambda, that would be extremly useful for creating readable sources. Usage could be something like: >>> res = [ f(i) for i in objects ] where: >>> def f(x): >>> #do something Hmm, this is actually a really interesting idea. Avoiding accidental namespace conflicts is certainly one of the advantages of using lambdas. This idea has the virtue of being able to do the same thing, but have full access to Python's function syntax and assignment statements in the 'expression local' suite. In fact, any subexpressions in a complicated expression can be extracted and named for clarity without fear of screwing up the containing namespace, which would be an enormous boon for software maintainers. It also allows the necessary but uninteresting setup for an expression to be moved "out of the way", bringing the expression that does the real work to prominence. From the interpreter's point of view, the meaning would probably be something like: namespace = locals() exec where_suite in globals(), namespace exec statement in globals(), namespace res = namespace["res"] del namespace Making the 'where' clause part of the grammar for the assignment statement should be enough to make the above example parseable, too. The clause might actually make sense for all of the simple statement forms in the grammar which contain an expression: expression statement assignment statement augmented assignment statement del statement print statement return statement yield statement raise statement exec statement The clause really isn't appropriate for break, continue, import or global statements, as they don't contain any expressions :) For compound statements, a where clause probably isn't appropriate, as it would be rather unclear what the where clause applied to. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: switching an instance variable between a property and a normal value
Steven Bethard wrote: I'd like to be able to have an instance variable that can sometimes be accessed as a property, and sometimes as a regular value, e.g. something like: If you want the behaviour to be switchable per-instance, you have to go the route of always running through the property machinery, since property() creates a descriptor on the class, not the instance. On the other hand, if you want to switch the behaviour of every instance, then making usevalue and usefunc class methods may give you some mileage. I'm rather curious about your use case, though. . . Here's a different way of using the property machinery, too: Py> class C(object): ... def __init__(self): ... self._use_val = None ... def useval(self, x): ... self._val = x ... self._use_val = True ... def usefunc(self, func, *args, **kwds): ... self._func, self._args, self._kwds = (func, args, kwds) ... def _get(self): ... use_val = self._use_val ... if use_val is None: ... raise AttributeError('x') ... if use_val: ... return self._val ... else: ... return self._func(*self._args, **self._kwds) ... x = property(_get) ... Py> c = C() Py> c.useval(4) Py> c.x 4 Py> c.usefunc(list) Py> c.x [] -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: python3: 'where' keyword
Nick Coghlan wrote: It also allows the necessary but uninteresting setup for an expression to be moved "out of the way", bringing the expression that does the real work to prominence. Killer app for this keyword: class C(object): x = property(get, set) where: def get(self): return "Silly property" def set(self, val): self.x = "Told you it was silly" Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: python3: 'where' keyword
Paul Rubin wrote: the suite has its own scope so any variable created there is local to the suite plus the following statement. The scope vanishes after the statement. The second part of the idea is to give the statement greater prominence and 'hide' the uninteresting setup (the contents of the where clause). Putting the statement of interest after the where suite still gives the benefits of avoiding namespace clutter, but doesn't really help readability (it makes it worse, if you ask me). Particularly, what if the next statement is a compound statement? Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: switching an instance variable between a property and a normal value
Steven Bethard wrote: where I also accept *args and **kwds when the default value is to be called. It's certainly doable with a flag, but note that I have to check the flag every time in both __getitem__ and setdefault. Alternatively, always use a function for the default value, and set _func to lambda: x when working by value :) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Other notes
Bengt Richter wrote: IOW, I think there is a fix: keep tokenizing greedily and tokenize floating point as a sequence of integers and operators, and let be translated by the compiler to floating point, and be translated to the appropriate generator expression implementation. That would be: -> float( + "." + ) -> getattr(int(), ) -> xrange(, ) However, the problem comes when you realise that 1e3 is also a floating point literal, as is 1.1e3. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: python3: accessing the result of 'if'
Carl Banks wrote: Right. But you know that as soon as you add this to simple expressions, a bunch of people are going to come here whining about how they don't get to use where with if-expressions. Frankly, they might have a point here. Although we have replacing lambda expressions on our minds, I have in mind a different problem that a where-statement would solve perfectly. But it would have to be used with an if-expression. I have a different suggestion for this. 'as' is used for renaming in import statements. 'as' will be used for exception naming in Python 3k. So let's use it for expression naming in 'if' statements, too. if someregexp.match(s) as m: # blah using m elif someotherregexp.match(s) as m: # blah using m Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: "A Fundamental Turn Toward Concurrency in Software"
Steve Horsley wrote: But my understanding is that the current Python VM is single-threaded internally, so even if the program creates multiple threads, just one core will be dividing its time between those "threads". Not really. The CPython interpreter does have a thing called the 'Global Interpreter Lock' which synchronises access to the internals of the interpreter. If that wasn't there, Python threads could corrupt the data structures. In order to do anything useful, Python code must hold this lock, which leads to the frequent misapprehension that Python is 'single-threaded'. However, the threads created by the Python threading mechanism are real OS threads, and the work load can be distributed between different cores. In practice, this doesn't happen for a pure Python program, since any running Python code must hold the interpreter lock. The Python threads end up getting timesliced instead of running in parallel. Genuine concurrency with pure Python requires running things in separate processes (to reliably get multiple instances of the Python interpreter up and running). Python threads are mainly intended to help deal with 'slow' I/O operations like disk and network access - the C code that implements those operations *releases* the GIL before making the slow call, allowing other Python threads to run while waiting for the I/O call to complete. This behaviour means threading can give *big* performance benefits on even single-CPU machines, and is likely to be the biggest source of performance improvements from threading. However, on multi-processor machines, it is also handy if a CPU-intensive operation can be handled on one core, while another core keeps running Python code. Again, this is handled by the relevant extension releasing the GIL before performing its CPU-intensive operations and reacquiring the GIL when it is done. So Python's concurrency is built in a couple of layers: Python-level concurrency: Multiple processes for true concurrency Time-sliced concurrency within a process (based on the GIL) C-level concurrency: True concurrency if GIL is released when not needed In some cases, problems with multi-threading are caused by invocation of extensions which don't correctly release the GIL, effectively preventing *any* other Python threads from running (since the executing extension never releases it). As an example, I frequently use SWIG to access hardware API's from Python. My standard 'exception translator' (which SWIG automatically places around every call to the extension) now looks something like: %exception { Py_BEGIN_ALLOW_THREADS try { $action } except (...) { Py_BLOCK_THREADS SWIG_exception(SWIG_RuntimeError, "Unexpected exception") } Py_END_ALLOW_THREADS } The above means that every call into my extension releases the GIL automatically, and reacquires it when returning to Python. I usually don't call the Python C API from the extension, but if I did, I would need to reacquire the GIL with PyGILState_Ensure() before doing so. Without those threading API calls in place, operations which access the hardware always block the entire program, even if the Python program is multi-threaded. See here for some more info on Python's threading: http://www.python.org/doc/2.4/api/threads.html Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: python3: 'where' keyword
Bengt Richter wrote: On Sat, 08 Jan 2005 16:42:16 +1000, Nick Coghlan <[EMAIL PROTECTED]> wrote: And, is the whole thing after the '=' an expression? E.g., x = ( foo(x) where: x = math.pi/4.0 ) where: def foo(x): print 'just for illustration', x or is this legal? for y in ([foo(x) for x in bar] where: bar = xrange(5) ): baz(y) where: def baz(arg): return arg*2 Not trying to sabotage the idea, really, just looking for clarification ;-) Actually, I was conceiving this as an addition to the grammar for the relevant 'simple statements', rather than to the grammar for expressions. Using the assignment statement as the ongoing example: Current: assignment_stmt ::= (target_list "=")+ expression_list augmented_assignment_stmt ::= target augop expression_list New: assignment_stmt ::= (target_list "=")+ expression_list [where_clause] augmented_assignment_stmt ::= target augop expression_list [where_clause] where_clause ::= "where" ":" suite So the expressions in existing compound statements (for, while, if, elif) would be out of luck. You could conceivably add the 'where' clause to the end of those as well, to give statement local variables that apply to the whole compound statement: for y in [foo(x) for x in bar]: baz(y) where: meaningful_name = xrange(5) def baz(arg): return arg * 2 This would only be appropriate for short loops - for long loops, the 'where' clause gets *too* hidden. Keeping the grammar simple might favour making the addition higher in the parse tree: Current: statement ::= stmt_list NEWLINE | compound_stmt New: statement ::= (stmt_list NEWLINE | compound_stmt) [where_clause] where_clause ::= "where" ":" suite However, doing it that way allows nonsense like this: pass where: print "This is just plain silly!" That would be something to be thrashed out in a PEP, though. The name 'statement local variables' also gave me an idea for a rough implementatation strategy. where: would be equivalent to: def stmt_with_locals(): stmt_with_locals() For the assignment versions, the behaviour would be: def assignment_with_locals(): return = assignment_with_locals() Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: python3: 'where' keyword
Paul Rubin wrote: What would be the advantage of that over this? . x = sqrt(a) + sqrt(b) where: . a = 2.0 . b = 3.0 The idea of "where" is to allow re-using variable names instead of having to keep track of which ones are in use. I just tried to give a very simple example of how you might do that more than once in a statement. I think having to keep the names unique within the statement you are currently writing is a reasonable request :) Cheers, Nick -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: python3: accessing the result of 'if'
Carl Banks wrote: What if the condition you wanted to test wasn't the same as the thing you want to save? In other words, how would you convert this? . where: . m = something() . if m > 20: . do_something_with(m) Yeah, this problem eventually occurred to me as well. However, I think a little utility function can help solve it: def test(val, condition): if condition(val): return val else: return None if test(something(), lambda x: x < 10) as m: print "Case 1:", m elif test(something(), lambda x: x > 20) as m: print "Case 2:", m else: print "No case at all!" If we were to use a where clause instead, it looks like: if test(something(), less_than(10)) as m: print "Case 1:", m elif test(something(), more_than(20)) as m: print "Case 2:", m else: print "No case at all!" where: def less_than(y): def lt(x): return x < y return lt def more_than(y): def gt(x): return x > y return lt This is an example of why I don't think where clauses would completely eliminate the utility of deferred expressions. Here's a version using my preferred syntax from the AlternateLambdaSyntax page: if test(something(), (def x < 10 from x)) as m: print "Case 1:", m elif test(something(), (def x > 20 from x)) as m: print "Case 2:", m else: print "No case at all!" Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: mysterious buggy behavior
Sean McIlroy wrote: While fiddling with a little script I ran into a problem that baffles me completely. Maybe I'm missing something completely obvious, and somebody out there can diagnose the problem at a glance. Anyway, that's the hope. Here's the code (it plays tic tac toe): You need to be a little more explicit than simply saying "something goes wrong". Exception? Wrong move? What? Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: python3: 'where' keyword
Paul Rubin wrote: AdSR <[EMAIL PROTECTED]> writes: Killer app for this keyword: class C(object): x = property(get, set) where: def get(self): return "Silly property" def set(self, val): self.x = "Told you it was silly" Hey, this is super-elegant! Heh, even further: z = C() where: class C(object): ... Lets you make anonymous classes and singleton objects. Here's another nice one if 'where' is added to compound statements as well: @dbc(pre, post) def foo(): pass where: def pre(): pass def post(): pass Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: python3: 'where' keyword
Paul Rubin wrote: Nick Coghlan <[EMAIL PROTECTED]> writes: I think having to keep the names unique within the statement you are currently writing is a reasonable request :) Um, you could say the same thing about the function, the module, etc. ;) And, indeed, that is what Python currently says. When writing code, the relevant namespaces are the builtins, the module, any containing functions and the current function. Inadvertent conflicts with any of those can have surprising side effects. The idea of 'where' is to push that down one level, and allow a namespace to be associated with a single statement. Trying to push it a level further (down to expressions) would, IMO, be a lot of effort for something which would hurt readability a lot. Compare: x = sqrt(a) + sqrt(b) where: a = 2.0 b = 3.0 This brings the operation we care about (add the sqrt's of 2 and 3) right up front. A folding code editor could actually hide the details quite easily. We can look inside the statement if we want to know what x & y actually are. Versus: x = (sqrt(a) where: a = 2.) \ + sqrt (a) where: a = 3. We haven't gotten rid of anything here - all the stuff we're interested in clearing out of the way is still embedded in the middle of our statement. Also not insignificantly, we're trying to put a suite inside an expression, which will be rejected for all the reasons that have kept lambda restricted to a single expression despite numerous complaints over time. Now, nothing in the idea of a statement local namespace actually *rules out* the prospect of an expression local namespace, so it could be added at a later date. However, doing so would require some actual use cases, and an expression-friendly syntax. Perhaps something that involves providing the namespace directly, like: x = (sqrt(a) where (a=2.0)) + (sqrt(b) where (a=3.0)) It seems to make more sense to try for statement local namespaces *first*, and then see if expression local namespaces are worth it. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: python3: accessing the result of 'if'
Carl Banks wrote: I'm sorry, I really can't agree that this helper function "solves" it. IMO, it's a workaround, not a solution. And, if I may be frank, it's a pretty ugly one. Heck, I thought it was ugly and I wrote it :) So in reality, I'd continue to use the nested-if approach that works right now if I wanted access to part of the condition instead of the whole thing. However, being able to bind a name to the conditions would be handy for many cases. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: python3: 'where' keyword
Alex Martelli wrote: I wonder if 'with', which GvR is already on record as wanting to introduce in 3.0, might not be overloaded instead. Perhaps we could steal 'using' from the rejected decorator syntax. x = [f(x) for x in seq] using: def f(x): return x * x Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: python3: 'where' keyword
Andrey Tatarinov wrote: sorry, I used "expression" carelessly. I mean that >>> print words[3], words[5] is a single expression (and that would be in Python 3, when print would be subtituted with write()/writeln()). 'statement' is the appropriate word in Python's grammar. And I don't think we'd actually have to wait for Python 3 for this, we'd just have to do the __future__ dance in order to introduce the new keyword. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: python3: 'where' keyword
Paul Rubin wrote: Nick Coghlan <[EMAIL PROTECTED]> writes: Trying to push it a level further (down to expressions) would, IMO, be a lot of effort for something which would hurt readability a lot. I think we should just try to do things in a simple and general way and not try to enforce readability. Trying to embed a suite in a Python expression is anything but simple, though :) Attaching an optional local suite to every statement can actually be done quite cleanly (so long as the parser is up to the job). Doing the substitutions on the whole statement using distinct names will work just as well as doing the substitutions directly on the subexpressions using duplicate names, and moves the 'unimportant' bits after the whole thing, instead of sprinkling them throughout the statement. It seems a statement based approach would be: 1. Easier to read 2. Less work to implement 3. Consistent with the rest of the language 4. Works for any statements, including compound statements The only disadvantage relative to expression local namespaces would be that you would still have to watch for namespace conflicts within the statement - and I have grave doubts about the structure of any statement which has that problem. If statement local namespaces were pursued, I would see expression local namespaces as something which hurt readability a lot without providing any new functionality. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Statement local namespaces summary (was Re: python3: 'where' keyword)
Andrey Tatarinov wrote: So it seems that people loved the idea of 'where' keyword, may be it's time to think about PEP draft? I appreciate any help (cause my english is not that good =)). There's certainly a PEP in the idea. Here's a summary of what we have so far in this thread in a PEPish format. Any reference implementation should definitely wait for the compiler upgrade to hit the main development branch in CVS though (or be based on ast-branch if someone gets inspired in the interim). Cheers, Nick. Abstract The proposal is to add the capacity for statement local namespaces to Python. This allows a statement to be placed at the current scope, while the statement's 'setup code' is indented after the statement:: with: The main benefit is the avoidance of namespace pollution in the outer scope. Sections of the statement can be extracted and named in the statement's local namespace without any inadvertent side effects due to name conflicts. The second benefit is that it can improve readability by allow easy grouping of the setup code for a given expression (see the examples section) Thirdly, it provides full-fledged effectively anonymous functions (just define them in the statement's local namespace). The idea is inspired by Haskell's where clause, as posted to python-list by Andrey Tatarinov. Alex Martelli suggested using the 'with' keyword. Grammar Change -- Current:: statement ::=stmt_list NEWLINE | compound_stmt New:: statement ::=(stmt_list NEWLINE | compound_stmt) [local_namespace] local_namespace ::= "with" ":" suite Semantics - The code:: with: translates to:: def unique_name(): unique_name() Assignment statements (including augmented assignment) require special handling. The original assignment statement is translated to a return statement in the inner scope:: with: translates to:: def unique_name(): return unique_name() Function and class definitions will also require special casing, so that the created function or class is returned from the inner-scope and the name bound correctly in the outer scope. Note that calling locals() inside a statement with a local namespace will refer to the statement's locals, rather than those of the containing function. Keyword Choice -- The original suggestion on python-list used the 'where' keyword. Alex Martelli pointed out that this could be misleading for anyone with expectations based on SQL's WHERE clause. He suggested 'with' as an alternative spelling, as 'with' is already planned as a keyword for Python3k. Even for with clauses associated with compound statements, there should be no ambiguity, given that the with statement will have an expression between it and the colon, while the with clause does not. Open Issues --- Is it actually possible to make it work? Keyword choice Should the clause be allowed on any statement (as described), or restricted to ones where it "makes sense"? Examples # Statement local functions (from Andrey Tatarinov) # aka How to cope if lambda goes away :) res = [ f(i) for i in objects ] with: def f(x): #do something # Declaring properties (from Nick Coghlan) class C(object): x = property(get, set) with: def get(self): pass def set(self, value): pass # Design by contract (from Nick Coghlan) @dbc(pre, post) def foo(): pass with: def pre(): pass def post(): pass # Singleton classes (from Paul Rubin) C = C() with: class C: pass # Complex default values (from Carlos Ribeiro) def f(x=default()): pass with: def default(): pass -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Long strings as function parameters
Jeremy Bowers wrote: I had thought there was an obvious class in the standard library to assist with this, but I must have been wrong. buffer is the closest current contender, but I believe it's on the outer due to some problems with its implementation. I think the intention is to eventually have a 'bytes' type, but I don't recall if that was going to be mutable or immutable. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: static compiled python modules
Thomas Linden wrote: How can I tell python to use the compiled in modules and not try to load them from outside? http://www.python.org/dev/doc/devel/api/importing.html Take a look at the last three entries about registering builtin modules. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: python3: accessing the result of 'if'
Steve Holden wrote: Excuse me, these are supposed to be IMPROVEMENTS to Python? I think it's more messing around before coming to the conclusion that, of the many things that 'where' helps with, this sure as hell ain't one of them :) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: python3: 'where' keyword
Andrey Tatarinov wrote: And about examples for usage "where" keyword reading http://manatee.mojam.com/~skip/python/fastpython.html I understand that almost every example should use that keyword =) I suspect polluting the outer namespace would still be faster, since Python wouldn't have to create the extra level of scoping all the time. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Statement local namespaces summary (was Re: python3: 'where' keyword)
Duncan Booth wrote: Nick Coghlan wrote: Grammar Change -- Current:: statement ::=stmt_list NEWLINE | compound_stmt New:: statement ::=(stmt_list NEWLINE | compound_stmt) [local_namespace] local_namespace ::= "with" ":" suite Semantics - The code:: with: translates to:: def unique_name(): unique_name() Your proposed grammar change says that you need a newline after the statement: True, and not really what I intended. However, it does highlight the fact that statement lists haven't been considered in the discussion so far. If statement lists are permitted, getting the right targets bound in the containing scope is going to be fun for things like this: a = b = 2; c = 3; print a + b with: pass (Probably not impossible though - e.g. it may be feasible to figure out all the bindings that have to happen and return an appropriate tuple from the inner scope for binding in the outer scope) It might also be somewhat confusing as to exactly which statements are covered by the local scoping. (All of them would be, but you could be forgiven for assuming it was just the last one) I think the following would work as a version of the grammar which permits local namespaces for statement lists: statement ::= (stmt_list (NEWLINE | local_namespace)) | (compound_stmt [local_namespace]) local_namespace ::= "with" ":" suite Disallowing local namespaces for statement lists would suggest something like this: statement ::= (simple_stmt (NEWLINE | ";" stmt_list NEWLINE | local_namespace) ) | (compound_stmt [local_namespace]) local_namespace ::= "with" ":" suite I'm pretty sure the permissive version is legal for the CPython parser, and I think the somewhat baroque structure I've used for the restrictive version makes it legal, too. Disallowing local namespaces for statement lists might be a good place to start, since it gives an easier target for a first implementation. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Statement local namespaces summary (was Re: python3: 'where' keyword)
Nick Coghlan wrote: Disallowing local namespaces for statement lists would suggest something like this: statement ::= (simple_stmt (NEWLINE | ";" stmt_list NEWLINE | local_namespace) ) | (compound_stmt [local_namespace]) local_namespace ::= "with" ":" suite Corrected version of the above to avoid an unintended syntax change: statement ::= (simple_stmt (NEWLINE | ";" [stmt_list] NEWLINE | local_namespace) ) | (compound_stmt [local_namespace]) local_namespace ::= "with" ":" suite (The original version incorrectly prohibited a trailing semi-colon for a single statement) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: windows mem leak
Bob Smith wrote: Peter Hansen wrote: Bob Smith wrote: Attached is the code. Run it yourself and see. You too Peter. Be gentle with me, this was my first attempt with threads. Thanks, Bob, and I will, but not before you answer some of my questions. I had good reasons to ask them, one of which is that I don't feel like wasting my time if, for example, you are using an older version of Python that *did* have a memory leak. 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] Perhaps you could try using the new subprocess module, instead of using os.popen directly. A fair amount of work went into making the Windows implementation of that module as solid as the *nix implementation, whereas there may still be issues with direct os.popen calls (as Roel's investigation suggests). Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Python3: on removing map, reduce, filter
Terry Reedy wrote: "Andrey Tatarinov" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] How does GvR suggestions on removing map(), reduce(), filter() While GvR *might* prefer removing them completely on any given day, I think moving them to a functional module, as others have suggested and requested, is currently more likely. I believe that GvR has indicated at times that this would be an acceptible compromise. I am one of those who think the list of builtins is currently too long to be easily grasped and should be shrunk. Heh. When PEP 309 hits CVS (with functional.partial), maybe it can grow aliases for the three of them so people can get used to the idea. It might keep partial from getting too lonely. . . :) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Speed revisited
John Machin wrote: My wild guess: Not a common use case. Double-ended queue is a special purpose structure. As Kent said, the suggestion of making index 0 insertions and deletions on lists more efficent was made, and the decision was to leave list alone and provide collections.deque instead. This let deque sacrifice some of list's flexibility in favour of increased speed. Appropriate parts of the core which needed a FIFO were then updated to use the new data type, while everything else continues to use lists. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Statement local namespaces summary (was Re: python3: 'where' keyword)
Nick Coghlan wrote: Semantics - The code:: with: translates to:: def unique_name(): unique_name() Bleh. Not only was my proposed grammar change wrong, my suggested semantics are wrong, too. Raise your hand if you can see the problem with applying the above semantics to the property descriptor example. So I think the semantics will need to be more along the lines of "pollute the namespace but mangle the names so they're unique, and the programmer can *act* like the names are statement local". This will be much nicer in terms of run-time performance, but getting the locals() builtin to behave sensibly may be a challenge. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Statement local namespaces summary (was Re: python3: 'where' keyword)
Nick Coghlan wrote: Nick Coghlan wrote: Semantics - The code:: with: translates to:: def unique_name(): unique_name() Bleh. Not only was my proposed grammar change wrong, my suggested semantics are wrong, too. Raise your hand if you can see the problem with applying the above semantics to the property descriptor example. Eh, never mind. The following works today, so the semantics I proposed are actually fine. (This is exactly the semantics proposed for the property example) Py> class C(object): ... def _x(): ... def get(self): ... print "Hi!" ... def set(self, value): ... print "Hi again!" ... def delete(self): ... print "Bye" ... return property(get, set, delete) ... x = _x() ... Py> C.x Py> C().x Hi! Py> C().x = 1 Hi again! Py> del C().x Bye Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Statement local namespaces summary (was Re: python3: 'where' keyword)
Andrey Tatarinov wrote: afair you told yourself that var = where: translates to: def unique_name(): return var = unique_name() in this case class gets unique_name() function? is it that bad? No, I wasn't thinking clearly and saw problems that weren't there. However, you're right that the semantic definition should include unbinding the unique name after the statement finishes. E.g. for assignments: def unique_name(): return = unique_name() del unique_name anyway I'd prefer to change semantics deeper. adding new statement-only scope and adding our suite-definitions there. A new scope essentially *is* a nested function :) My main purpose with the nested function equivalent is just to make the intended semantics clear - whether an implementation actually _does_ things that way is immaterial. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Statement local namespaces summary (was Re: python3: 'where' keyword)
Andrey Tatarinov wrote: I think using 'with' keyword can cause some ambiguity. for example I would surely try to write >>> x = a+b with self: >>> b = member and using with at the end of block brings more ambiguity: >>> stmt1() >>> stmt2() >>> with self: >>> member = stmt3() compare to: >>> stmt1() >>> stmt2() >>> with: >>> variable = stmt3() a way different semantics with just one word added/deleted. Except that for a "with :" block, attributes of the expression must be preceded by a dot: Py> stmt1() Py> stmt2() Py> with self: ....member = stmt3() The advantages of the 'with' block are that 'self' is only looked up once, and you only need to type it once. The leading dot is still required to disambiguate attribute references from standard name references. Despite that, I think you are right that the ambiguity is greater than I first thought. Correct code is reasonably easy to distinguish, but in the presence of errors it is likely to be unclear what was intended, which would make life more difficult than it needs to be. However, I still agree with Alex that the dual life of "where" outside of Python (as an 'additional definitions' clause, as in mathematics, and as a 'conditional' clause, as in SQL), and the varied background of budding Pythoneers is a cause for concern. 'in' is worth considering, as it is already used by Python at least once for declaring use of a namespace (in the 'exec' statement). However, I suspect it would suffer from ambiguity problems similar to those of 'with' (consider " in " and " in: "). There's also the fact that the statement isn't *really* executed in the inner namespace - any name binding effects are seen in the outer scope, whereas 'exec x in dict' explicitly protects the containing namespace from alteration. So of the four keywords suggested so far ('where', 'with', 'in', 'using'), I'd currently vote for 'using' with 'where' a fairly close second. My vote goes to 'using' because it has a fairly clear meaning ('execute the statement using this extra information'), and doesn't have the conflicting external baggage that 'where' does. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Securing a future for anonymous functions in Python
Tim Peters wrote: LIke it or not, it doesn't seem as strained as trying to pile more gimmicks on Python expressions. Some of us are piling gimmicks on statements, not expressions :) And I'm looking for out-of-order code execution as well as local namespaces, so the let/in syntax wouldn't help much. (The out-of-order execution is what really makes the property definition example pretty, IMO) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: "Architecture of Python" was removed ?
cr999 wrote: > I found the "Architecture of Python" ( http://wiki.cs.uiuc.edu/cs427/PYTHON > By Jim Jackson, Kar-Han Tan )is very useful for my understanding of the > Python's architecture. But I found the link is not link to that document > today. It seems that the document was removed. Who knows what happened? > > Does anyone here have a copy of that document? Or who can tell me what is the > email address of Jim Jackson or Kar-Han Tan. http://web.archive.org/web/2003101953/http://wiki.cs.uiuc.edu/cs427/PYTHON Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: else condition in list comprehension
Dan Bishop wrote: Luis M. Gonzalez wrote: Hi there, I'd like to know if there is a way to add and else condition into a list comprehension. I'm sure that I read somewhere an easy way to do it, but I forgot it and now I can't find it... for example: z=[i+2 for i in range(10) if i%2==0] what if I want i [sic] to be "i-2" if i%2 is not equal to 0? z = [i + (2, -2)[i % 2] for i in range(10)] For the specific case of +/- a number, (-1) ** x works, too: z = [i + 2 * ((-1) ** i) for i in range(10)] Not that I'm claiming it's particularly readable or anything. . . just that it works :) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: [Python-Dev] PEP 246, redux
Alex Martelli wrote: "I really wish the language had private inheritance because I'm using Abstract as a base just for code reuse" Funny you should say that. . . what about a __nonconformant__ entry that accepts a list of base classes that is used to indicate inheritance without a proper is-a relationship? So if the isinstance check succeeds and there is no '__nonconformant__' entry, then adapt() just returns the object. If, on the other hand, __nonconformant__ is supplied, then adapt() can check the list of base classes that remains after removing the entries in __nonconformant__ to see if the object would *still* be an instance a subtype, even after removing the noncomformant bases, and if it is, return it. Otherwise, continue on to the rest of the adaptation process. This should give a better idea what I mean: # The 'fast path' if isinstance(obj, protocol): if not hasattr(obj, "__nonconformant__"): return obj conformant_bases = set(obj.__bases__) - set(obj.__nonconformant__) for base in conformant_bases: if issubtype(base, protocol): return obj # Continue on with the other adaptation possibilities (including __conform__) Then you can get 'discreet' inheritance (you have the methods, but you don't brag about the fact) by writing: class Dubious(Abstract): __noncomformant__ = [Abstract] # etc rather than: class Dubious(Abstract): def __comform__(self, protocol): if issubtype(protocol, Abstract): raise LiskovViolation # etc Regards, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Exception not captured
Miki Tebeka wrote: I get to the second "except" clause, and the printout is: /home/mikit/work/nightly/scm/common.py:3 /home/mikit/work/nightly/scm/common.py:3 False How is this possible? Is line 3 inside a function? Then the class will be recreated anew each time the function is run. Has common.py been reload()'ed at some point? Then previously imported modules may still have names bound to the old instances. And so forth. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: else condition in list comprehension
Luis M. Gonzalez wrote: Hi there, I'd like to know if there is a way to add and else condition into a list comprehension. I'm sure that I read somewhere an easy way to do it, but I forgot it and now I can't find it... for example: z=[i+2 for i in range(10) if i%2==0] what if I want i to be "i-2" if i%2 is not equal to 0? Hmm: z = [newval(i) for i in range(10)] using: def newval(x): if x % 2: return x - 2 else: return x + 2 Just some more mental twiddling relating to the thread on statement local namespaces. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Statement local namespaces summary (was Re: python3: 'where' keyword)
Nick Coghlan wrote: Semantics - The code:: with: translates to:: def unique_name(): unique_name() I've come to the conclusion that these semantics aren't what I would expect from the construct. Exactly what I would expect can't really be expressed in current Python due to the way local name bindings work. The main thing to consider is what one would expect the following to print: def f(): a = 1 b = 2 print 1, locals() print 3, locals() using: a = 2 c = 3 print 2, locals() print 4, locals() I think the least suprising result would be: 1 {'a': 1, 'b': 2} # Outer scope 2 {'a': 2, 'c': 3} # Inner scope 3 {'a': 2, 'b': 2, 'c': 3} # Bridging scope 4 {'a': 1, 'b': 2} # Outer scope In that arrangement, the statement with a using clause is executed normally in the outer scope, but with the ability to see additional names in its local namespace. If this can be arranged, then name binding in the statement with the using clause will work as we want it to. Anyway, I think further investigation of the idea is dependent on a closer look at the feasibility of actually implementing it. Given that it isn't as compatible with the existing nested scope structure as I first thought, I suspect it will be both tricky to implement, and hard to sell to the BDFL afterwards :( Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: else condition in list comprehension
Andrey Tatarinov wrote: I presume the point of this is to avoid polluting the local namespace with "newval". I further presume you also have plans to do something about "i"? ;-) no, the point is in grouping definition of newval() with place where it is used. I'd have said the point was both :) But yeah, unfortunately the 'leaking list comp' problem won't be fixed in the 2.x series due to the compatibility problem. Fortunately, generator expressions didn't inherit the issue. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Statement local namespaces summary (was Re: python3: 'where' keyword)
Bengt Richter wrote: Problems? (Besides NIH, which I struggle with regularly, and had to overcome to accept Tim's starting point in this ;-) The ideas regarding creating blocks whose name bindings affect a different scope are certainly interesting (and relevant to the 'using' out-of-order execution syntax as well). Out-of-order execution appeals to me, but the ability to flag 'hey, this is just setup for something I'm doing later' might be a reasonable alternative (particularly with the affected names highlighted on the first line). As Jeff pointed out, it would be significantly less surprising for those encountering the construct for the first time. Folding code editors would be able to keep the setup clause out of the way if you really wanted to hide it. On the other hand, it might be feasible to construct a virtually identical out-of-order two suite syntax, similar to the mathematical phrasing "let f = c/lambda where f is the frequency, c is the speed of light and lambda is the wavelength". Either way, you've convinced me that two suites (and a new compound statement), as well as specifying which names can be rebound in the containing scope, is a better way to go than trying to mess with the definition of Python statements. On keywords, while 'let' is nice for assignments, I find it just doesn't parse properly when I put function or class definitions in the clause. So, I'll swap it for 'use' in the examples below. The statement could then be read "use these outer bindable names, and this additional code, in this suite". YMMV, naturally. Let's consider some of the examples given for 'where' using an in-order let/in type syntax (the examples only bind one name at a time, but would allow multiple names): # Anonymous functions use res: def f(x): d = {} exec x in d return d in: res = [f(i) for i in executable] # Declaring properties class C(object): use x: def get(self): print "Demo default" def set(self, value): print "Demo default set" in: x = property(get, set) # Design by contract use foo: def pre(): pass def post(): pass in: @dbc(pre, post) def foo(): pass # Singleton classes use C: class _C: pass in: C = _C() # Complex default values use f: def default(): return "Demo default" in: def f(x=default()): pass They actually read better than I expected. Nicely, the semantics of this form of the syntax *can* be articulated cleanly with current Python: use : in: as equivalent to: def __use_stmt(): def _in_clause(): return return _in_clause() __use_stmt_args = {} = __use_stmt() del __use_stmt Those semantics don't allow your switch statement example, though, since it doesn't use any magic to write to the outer scope - it's just a normal return and assign. However, I don't think starting with these semantics would *preclude* adding the ability to name the second block at a later date, and make the name rebinding part of executing that block - the standard usage doesn't really care *how* the names in the outer scope get bound, just so long as they do. Whether I think that's a good idea or not is an entirely different question :) Another aspect to consider is whether augmented assignment operations in the inner-scopes should work normally - if so, it would be possible to alter the semantics to include passing the existing values as arguments to the inner scopes. Moving on to considering a two-suite out-of-order syntax, this would have identical semantics to the above, but a syntax that might look something like: as : using: # Anonymous functions as res: res = [f(i) for i in executable] using: def f(x): d = {} exec x in d return d # Declaring properties class C(object): as x: x = property(get, set) using: def get(self): print "Demo default" def set(self, value): print "Demo default set" # Design by contract as foo: @dbc(pre, post) def foo(): pass using: def pre(): pass def post(): pass # Singleton classes as C: C = _C() using: class _C: pass # Complex default values as f: def f(x=default()): pass using: def default(): return "Demo default" Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: python and macros (again) [Was: python3: 'where' keyword]
Paul Rubin wrote: Come on, that is vacuous. The claim was "expressions are not statements". But it turns out that expressions ARE statements. The explanation is "well, that's because they're expression statements". And there is no obvious case of an expression that can't be used as a statement. So it's not inherently obvious that there needs to be any kind of statement that can't be used as an expression. It's just an artifact. Whether the artifact is a desirable one is a matter of discussion. No, it's entirely to do with building a readable language that uses significant whitespace to delineate scope. Simple statements are not allowed to contain other statements, but they are allowed to contain expressions. In their most degenerate form, ALL they contain is a *single* expression (e.g. a function call). That expression itself is still not a statement, though. Think of it as the difference between value and [value]. Suites are sequences of statements. Compound statements are statements which incorporate a suite as part of their syntax. Python allows statements inside suites and suites inside compound statements. It also allows expressions inside statements and expressions inside expressions. The one thing it never ever does is allow a suite or a statement inside an expression, because doing so would utterly destroy the handling of significant white space. And that's why expressions are special in Python - they demarcate the boundary of the significance of whitespace. Within an expression, whitespace is insignificant. At the level of statements and suites, whitespace is extremely significant. So, precisely how should one go about cleanly embedding something that cares about whitespace into a context which doesn't care in the slightest? Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Class initialization from a dictionary, how best?
[EMAIL PROTECTED] wrote: t2 = Test(dictionary.get('a'), dictionary.get('b'), dictionary.get('c')) print t2 Try this: t2 = Test(**dictionary) This performs keyword argument expansion on the dictionary, matching the dictionary entries with the named arguments to the Test.__init__ function. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: python and macros (again) [Was: python3: 'where' keyword]
Antoon Pardon wrote: No I am applying set logic. Any string that is in the set of valid expressions is also in the set of valid statements. According to Python's grammar, this is not the case. It requires a NEWLINE or ";" token on the end to turn the expression into a statement. Actually appending either of those tokens means the string is no longer an expression. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Statement local namespaces summary (was Re: python3: 'where' keyword)
Nick Coghlan wrote: as equivalent to: def __use_stmt(): def _in_clause(): return return _in_clause() __use_stmt_args = {} = __use_stmt() del __use_stmt The more I think about this return-based approach, the less I like it. It could probably be made to work, but it just feels like a kludge to work around the fact that the only mechanisms available for altering the bindings of local names are assignment and definition statements. For class namespaces, getattr(), setattr() and delattr() work a treat, and globals() works fine for module level name binding. locals() is an unfortunate second class citizen, since it writes to it aren't propagated back to the executing frame. Programmatic interrogation of locals is fine, but update is impossible. What would be interesting is if locals() returned a dictionary whose __setitem__ method invoked PyFrame_LocalsToFast on the relevant frame, instead of a vanilla dictionary as it does now. Then locals()["x"] = foo would actually work properly. Notice that you can get this effect today, by using exec to force invocation of PyFrame_LocalsToFast: Py> def f(): ... n = 1 ... def g(outer=locals()): ...outer["n"] += 1 ... g() # Does not affect n ... print n ... exec "g()" # DOES affect n ... print n ... Py> f() 1 2 (The call to g() has to be inside the exec statement, since the exec statement evaluation starts with a call to PyFrame_FastToLocals). Assuming a writeable locals(), the semantics for the normal case are given by: def __use_stmt(__outer): __inner = locals() for name in : __outer[name] = __inner[name] __use_stmt(locals()) del __use_stmt And for the 'delayed execution' case: def __named_use_stmt(__outer): def __delayed_block(): __inner = locals() for name in : __outer[name] = __inner[name] return __delayed_block = __named_use_stmt(locals()) del __named_use_stmt Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: import keyword behaviour - performance impact if used multiple times?
neophyte wrote: Nick Coghlan wrote: > Is > this something to do with system modules being singletons? They aren't singletons in the GoF design pattern sense. However, Python's import machinery operates in such a way that it takes effort to get multiple version of the same module into memory at the same time (it *can* be done, but you have to work at it). Given that this is exactly what I want, how can you do it? If you just want to reload an existing module, use the builtin "reload" function. Getting multiple versions of a module into sys.modules at the same time isn't something I've ever actually wanted to do, but the following will do it: Py> import sys Py> sys.modules["sys1"] = sys Py> del sys.modules["sys"] Py> import sys Py> import sys1 Py> sys is sys1 False However: 1. Doing this at all is probably a bad idea (since you may end up duplicating objects that are meant to be unique within the process, and any C-extension code will still be shared between the multiple versions of the module) 2. Doing it to 'sys' like I just did is an even worse idea, since you *definitely* end up doing 1 :) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Why 'r' mode anyway?
Skip Montanaro wrote: Tim> "Plays well with others" was a strong motivator for Python's Tim> design, and that often means playing by others' rules. -- My vote for QOTW... Is it too late to slip it into the Zen of Python? It would certainly fit, and the existing koans don't really cover the concept. Its addition also seems fitting in light of the current PEP 246 discussion which is *all* about playing well with others :) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: How to del item of a list in loop?
Reinhold Birkenfeld wrote: Addition: In Py2.4, I can't find a problem with for i in reversed(lst) Some poking around suggests it's fine - and it avoids copying the list. Don't delete anything earlier in the list than the current element though (which list.remove() will do quite happily when data is duplicated). However, there will still be data movement costs when deleting elements from the middle of the list. In addition, remove() itself has to do a linear search for the value being removed. An off-place solution based on a list comprehension is usually going to be your best performer - it's an O(n) operation, based on the size of the original list. The in-place mechanisms can turn out to be O(n**2) due to worst-case memory movement effects (lists don't allow gaps, so deleting items will usually trigger data movement). I think this is about the best you can do for an in-place version: for i, x in enumerate(reversed(lst)): if x == 2: del lst[-i] The effbot's version is still going to be faster though: lst = [x for x in lst if x != 2] Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Producer/consumer Queue "trick"
Paul Rubin wrote: Evan Simpson <[EMAIL PROTECTED]> writes: wakes up the producer, which has been blocked waiting to add a board to the Queue. It sets about generating the next board, and the consumer doesn't get to run again until the producer blocks again or is preempted. That's weird. Preemption should happen every few dozen milliseconds unless you've purposely increased the preemption delay. To me, it smells like a call into a C extension which isn't releasing the GIL before starting a time-consuming operation. After getting bitten by this a couple of times, I now make sure to release the GIL as part of my SWIG wrapper (since the code I'm wrapping knows nothing of Python, and sure as heck doesn't need to be holding the GIL!). Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: How to del item of a list in loop?
John Machin wrote: Nick Coghlan wrote: The effbot's version is still going to be faster though: lst = [x for x in lst if x != 2] Have you measured this? Nope. I'm going purely on the fact that it is O(n), and the in-place modification will always be worse than that. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: generator expressions: performance anomaly?
John Machin wrote: Background: There was/is a very recent thread about ways of removing all instances of x from a list. /F proposed a list comprehension to build the result list. Given a requirement to mutate the original list, this necessitates the assignment to lst[:]. I tried a generator expression as well. However while the listcomp stayed competitive up to a million-element list, the genexp went into outer space, taking about 20 times as long. The above timeit runs show a simpler scenario where the genexp also seems to be going quadratic. Comments, clues, ... please. Py> lc = [x for x in range(100)] Py> len(lc) 100 Py> ge = (x for x in range(100)) Py> len(ge) Traceback (most recent call last): File "", line 1, in ? TypeError: len() of unsized object It would be nice if unconditional ge's with known length inputs propagated __len__, but that is not currently the case. There's a similar performance glitch associated with constructing a tuple from a generator expression (with vanilla 2.4, detouring via list is actually faster) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: (objects as) mutable dictionary keys
Antoon Pardon wrote: What are good arguments or bad and how much weight they have depends on the person and on the circumstances. So a simple rule like: Never use a mutable as a key in a dictionary will sometimes not be the best solution. True - but I think a purely identity based dictionary *is* the solution for most such cases where the standard Python rule doesn't apply :) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Executing a script created by the end user
Craig Howard wrote: I am working on a python project where an object will have a script that can be edited by the end user: object.script If the script is a simple one with no functions, I can easily execute it using: exec object.script Take a look at the execfile builtin. But if the object script is a bit more complicated, such as the example below, my approach does not work: Alternatively: Py> exec """ ... def main(): ... hello1() ... hello2() ... ... def hello1(): ... print 'hello1' ... ... def hello2(): ... print 'hello2' ... """ Py> main() hello1 hello2 'exec' is quite happy to deal with newlines and indenting. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: getting a class attribute using a keyword argument
Guy Robinson wrote: Hello, I have a list of class instances. I wish to get the appropriate class attribute in each class instance depending on a SINGLE keyword in the calling class. Py> help(getattr) Help on built-in function getattr in module __builtin__: getattr(...) getattr(object, name[, default]) -> value Get a named attribute from an object; getattr(x, 'y') is equivalent to x.y. When a default argument is given, it is returned when the attribute doesn't exist; without it, an exception is raised in that case. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: [perl-python] 20050117, filter, map
Steve Holden wrote: As I may have mentioned before, egotism can be the only possible reason. I'd merely figured it as a textbook case of trolling - attention seeking behaviour, most likely indicative of a lack of self-esteem, rather than the reverse. Still, he does at least keep the [perl-python] mailing list tag, so automatic filtering isn't that difficult. It is an unfortunate shame that his consideration doesn't extend to removing the general Perl and Python discussion groups from his recipients list. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: [Python-Dev] Getting rid of unbound methods: patch available
Guido van Rossum wrote: What do people think? (My main motivation for this, as stated before, is that it adds complexity without much benefit.) Something important that came up in my response to Marc-Andre: What about C method implementations which are relying on this typecheck and assuming that 'self' is (nearly) guaranteed to be of the correct type? The string object implementation is where I noticed this assumption being made, but I suspect it would be fairly widespread. Obviously, use of im_func can break such code already, but that's a far cry from having C.f skip the type check. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: extension module, thread safety?
Pierre Barbier de Reuille wrote: With the current CPython, it's very hard to mix Python and C in a multithreading application (with C-threads, not Python-threads). In fact I never really succeeded in that task because of that GIL ! I have a multi-thread application but every bit of Python code must be run into a Python thread. To be more precise, I wanted to be able to call Python code in response to some GUI events, and I didn't want to instanciate a new interpreter for I wanted to be able to access the environment of my main Python interpreter. I don't understand. This is what PyGILState_Ensure and PyGILState_Release are for - so C code can leave the GIL unlocked by default, and only grab it when they want to call into the C/Python API. Regards, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: generator expressions: performance anomaly?
Raymond Hettinger wrote: [Delaney, Timothy C] Nick's other suggestion - that genexps propagate __len__ - might still be interesting. Of course, it would only be applicable for unconditional genexps(i.e. no if clause). Length transparency for iterators is not as general as one would expect. I once spent a good deal of effort exploring where it made sense, and I was surprised to find that it only rarely works out. Length transparency is an unexpectedly thorny subject with many dead-ends which precludes a fully general solution such as that proposed by Nick. For a recap of my research, see the docstring for Lib/test/test_iterlen.py . """The situation slightly more involved whenever an object allows length mutation during iteration. """ Ouch. Nice understatement. It's rather unfortunate that we can't make use of the length information even when the source *doesn't* mutate, though. I'll have to think some more to see if I can come up with any concrete ideas for you to shoot down :) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: extension module, thread safety?
Pierre Barbier de Reuille wrote: Ok, I wondered why I didn't know these functions, but they are new to Python 2.4 ( and I didn't take the time to look closely at Python 2.4 as some modules I'm working with are still not available for Python 2.4). But if it really allows to call Python code outside a Python thread ... then I'll surely use that as soon as I can use Python 2.4 :) Thanks for the hint :) The Python 2.4 docs claim the functions were added in Python 2.3, even though they aren't documented in the 2.3.4 docs. The 2.3 release PEP (PEP 283) confirms that PEP 311 (which added these functions) went in. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: lambda
Antoon Pardon wrote: More specific the Decimal class is mutable and usable as dict key. It's *meant* to be immutable though. The fact that we used __slots__ instead of __setattr__ to implement the immutability, so you *can* overwrite the slot variables if you really want to is merely an artifact of the current Python implementation. The limited mutability bug will disappear in Python 2.5, so it's not a good example of a 'mutable' dict key (especially given that the only way to mutate it is to modify private variables directly). And, as I've stated previously, if the issue of sane mutable keys in dictionaries and sets really bugs you so much - implement identity_dict and identity_set in C and lobby for their inclusion in the collections module. On the more general point of "don't use mutable objects with non-identity based comparisons as dictionary keys", try teaching students for a while (or listen to those who have): When stating useful general principles, it is never, ever worth it to get into the quibbly little details about exceptions to the principle. If students ask, admit that they exist, but point out that the exceptions are rare, and not worth worrying about at that point in their learning. Only when one is aware of the reasons for a principle, can one be aware of good reasons not to follow it :) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Dictionary keys (again) (was Re: lambda)
Antoon Pardon wrote: A rule of thumb is context sensitive. If circumstances change, so do the rules of thumb. Principles have a broader field of application. IMO there is nothing principally wrong with using a mutable object as a dictionary key. But avoiding doing so is a good rule of thumb if you have a python-like implementation of a dictionary. For a 'mutable key' to make sense, the following: lst = [] dct = {l: "Hi!"} print dct[[]] print dct[lst] lst.append(1) print dct[[1]] print dct[lst] Should print: Hi Hi Hi Hi That's completely impractical though - for any sane implementation, at least one of the above print statements will throw a KeyError. Your example of a 'safe_dict' that copies mutable keys means that the final print statement is the one that fails. My suggestion of an "identity_dict" means that both of the same-value based lookups would fail. Notice that both of these approaches take a mutable key and make it immutable (either by holding the sole reference to a copy, or retrieving an immutable property of the mutable object). There's also your solution of "I promise not to mutate the key while it is in the dictionary". Workable, but opens the door to some entertaining bug hunts when the promise is broken. Which solution is the best default behaviour? Well, the Zen of Python states "In the face of ambiguity, refuse the temptation to guess". So that's the policy the builtin dict follows - it doesn't try to guess when to make a copy, or whether or not to use identity based semantics in the face of mutability. Instead, it raises an exception at key entry time, asking the programmer to clarify their intent. The principle is *certainly* that hash tables should contain immutable keys. Your own 'safe_dict' example implicitly concedes this point by making copies 'when required'. 'When required' for what? When required to preserve immutability, perhaps? In short, sane hash tables require immutable keys, and how mutable keys acquire the requisite immutability is going to be application dependent. Provision of a safe_dict or identity_dict would merely allow a programmer to state their intent at the time the container is created, rather than having to state it whenever keys are generated or referenced. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: lambda
Antoon Pardon wrote: > I can be wrong, but until now I have seen no indication that I was using mutable and immutable differently than other people. AFAICT we all refer to whether an object belongs to a mutable or immutable class. The difference is that when you take a copy of the key and stick it in the dictionary, such that the dictionary now holds the *sole* reference to that key, you have made that key *effectively* immutable. This is why no-one really batted an eyelid when you mentioned that mutable keys can be used safely by making a copy - doing so makes the key *effectively* immutable, and means that modifying the original key object (i.e. the application's copy, not the dict's copy), and reusing it to look up in the dictionary is likely to give a KeyError. These semantics would be understandably surprising to many users, and hence, are not supplied by default. Additionally, a dictionary's keys are accessible via its API. Accordingly, to preserve this 'effective immutability', making a copy on key input is insufficient - keys must be copied on *output* as well (that is, dict.keys, dict.items etc must return *copies* of the key objects, not the key objects themselves). Since there is no reliable way in Python to tell if an object is mutable or not (the closest equivalent is the presence of __hash__, which clearly can't be used in this example), this copying would need to be done for *every* object. Alternately, the dictionary can say to the API clients, "make an immutable copy and supply that instead. It is left to API clients to decide how best to make the immutable version". The API client copies the key once (to make the immutable version), and everyone lives happily ever after. For example: Py> class mylist(list): ... def __init__(self, arg): ... super(mylist, self).__init__(arg) ... self._tuple = None ... def frozen(self): ... if self._tuple is None: ... self._tuple = tuple(self) ... return self._tuple ... def unfreeze(self): ... self._tuple = None ... Py> x = mylist(range(10)) Py> x [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] Py> x.frozen() (0, 1, 2, 3, 4, 5, 6, 7, 8, 9) Py> x.append(10) Py> x.frozen() (0, 1, 2, 3, 4, 5, 6, 7, 8, 9) Py> x.unfreeze() Py> x.frozen() (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10) This is much safer than having a list subclass that implements __hash__, and still lets you avoid redundant copy operations. Nicely, so long as you don't unfreeze the object you used to key the dictionary, reusing the same object will always get you the correct dictionary entry, and two lists that compare equal at the time they are frozen will also get you the same dictionary entry. The equivalent tuples can be used for the lookup, too. I also don't want my values to change when I have sorted a list and still need to apply a number of algorithms that rely on that. Nobody seems to have problems with the possibility that the list items are mutable (and called such). OK, to make this comparison of sorted lists and dictionaries fair: Write a sorted_list class that is like a regular Python list, but maintains as an invariant that the list contents will stay sorted. See how well you go maintaining that invariant while allowing mutable objects in the list. The only way would be to copy them in when they're supplied, and copy them out again when you're done. Otherwise, there is no way the class can keep its promise. The performance will be lousy, since __setitem__ and __getitem__ will be making copies all the time. Alternatively, the class could declare itself to work reliably only with immutable objects. Performance will improve, since copies need only be made when an object *actually* changes (and the old immutable copy is deleted and the new version inserted in its place). Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: delay and force in Python
Will Stuyvesant wrote: The program below creates a stream with the numbers 1..995 and then filters the stream, keeping only the even numbers, and then prints the second number in the stream (implemented as the first number of the tail, just like in the 3.5 Section in the Wizard book). How's this: Py> from itertools import islice Py> print islice((x for x in xrange(1, 996) if x % 2 == 0), 1, 2).next() 4 Breaking it into pieces: Py> from itertools import islice Py> stream = (x for x in xrange(1, 996) if x % 2 == 0) Py> second_item = islice(stream, 1, 2).next() Py> print second_item 4 And most of the stream hasn't been consumed yet: Py> print stream.next() 6 Py> unconsumed = list(stream) Py> len(unconsumed) 494 And this version has no problem with recursion limits: Py> print islice((x for x in xrange(1, sys.maxint) if x % 2 == 0), 1, 2).next() 4 (xrange can't handle Python longs, unfortunately, so we *are* constrained by sys.maxint. However, since my machine only has half a gig of RAM, the above is still a damn sight quicker than the equivalent list comprehension would be!) Something else: this crashes with a "maximum recursion reached" . print stream_enumerate_interval(1,998) while this does not crash . print stream_enumerate_interval(1,900) this means Python has a maximum of something like 900 recursions? The CPython implementation is limited by the stack size allocated by the C runtime library. The exact recursion limit is platform dependent, but something around 1000 sounds fairly normal. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Dictionary keys (again) (was Re: lambda)
David Eppstein wrote: Yes, and what should the following do? lst1 = [1] lst2 = [2] dct = {lst1: "1", lst2: "2"} lst2[0]=1 lst1[0]=2 print dct[[1]] print dct[[2]] Provide yet another example for why mutable keys are almost guaranteed to result in suprising semantics :) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net -- http://mail.python.org/mailman/listinfo/python-list