Re: Guido rethinking removal of cmp from sort method
I suspect that this debate is a sink hole that I won't be able to escape from alive but... On 2 Apr 2011, at 19:29, harrismh777 wrote: In other words, does the PSF have a responsibility to maintain the L.sort(cmp= key= reverse=) interface for strictly *philosophical* principle based on established norms for *any* OOP language? (and) is there OOA&D expectation for this principle? No, there should be no expectation that Python 2.x interfaces be preserved in Python 3.x unless they have demonstrated utility. Furthermore, there should be no expectation that a particular interface survive for more than a few major Python versions. PEP-004 describes how deprecations are expected to proceed at module granularity. The rest of the thread is arguing for a *technical* determination for inclusion of the cmp= keyword... I am arguing (on the other hand) for a *philosophical* determination for inclusion of the cmp= keyword. Any argument along what you call "philosophical" grounds will not be successful. Technical (including aesthetic, convenience, etc.) arguments *may* be successful. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Guido rethinking removal of cmp from sort method
On 3 Apr 2011, at 16:22, geremy condra wrote: I think we're talking at cross purposes. The point I'm making is that there are lots of issues where popularity as a third party module isn't really a viable test for whether a feature is sufficiently awesome to be in core python. As part of determining whether I thought it was appropriate in this case I essentially just asked myself whether any of the really good and necessary parts of Python would fail to be readmitted under similar circumstances, and I think the answer is that very few would come back in. To me, that indicates that this isn't the right way to address this issue, although I admit that I lack any solid proof to base that conclusion on. This has been discussed a few times on python-dev. I think that most developers acknowledge that small-but-high-utility modules would not survive outside of the core because people would simple recreate them rather than investing the time to find, learn and use them. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Guido rethinking removal of cmp from sort method
On 3 Apr 2011, at 15:30, harrismh777 wrote: Brian Quinlan wrote: I suspect that this debate is a sink hole that I won't be able to escape from alive but... ... live long and prosper my friend. Something to consider is that OOP philosophy is technically one of the most aesthetic concepts in all of computer science--- with pure functional programming (haskel, erlang) as a close second... You can keep arguing that but you are tilting at windmills. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: "no variable or argument declarations are necessary."
Have those of you who think that the lack of required declarations in Python is a huge weakness given any thought to the impact that adding them would have on the rest of the language? I can't imagine how any language with required declarations could even remotely resemble Python. And if you want to use such a different language, wouldn't a different existing language better fit your needs...? Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: "no variable or argument declarations are necessary."
Paul Rubin wrote:
> Brian Quinlan <[EMAIL PROTECTED]> writes:
>
>>Have those of you who think that the lack of required declarations in
>>Python is a huge weakness given any thought to the impact that adding
>>them would have on the rest of the language? I can't imagine how any
>>language with required declarations could even remotely resemble
>>Python.
>
>
> Python already has a "global" declaration;
Which is evaluated at runtime, does not require that the actual global
variable be pre-existing, and does not create the global variable if not
actually assigned. I think that is pretty different than your proposal
semantics.
> how does it de-Pythonize the language if there's also a "local"
> declaration and an option to flag any variable that's not declared as
> one or the other?
Your making this feature "optional" contradicts the subject of this
thread i.e. declarations being necessary. But, continuing with your
declaration thought experiment, how are you planning on actually adding
optional useful type declarations to Python e.g. could you please
rewrite this (trivial) snippet using your proposed syntax/semantics?
from xml.dom import *
def do_add(x, y):
return '%s://%s' % (x, y)
def do_something(node):
if node.namespace == XML_NAMESPACE:
return do_add('http://', node.namespace)
elif node.namespace == ...
...
> There's been a proposal from none other than GvR to add optional
> static declarations to Python:
>
> http://www.artima.com/weblogs/viewpost.jsp?thread=85551
A few points:
1. making it work in a reasonable way is an acknowledged hard problem
2. it will still probably not involve doing type checking at
compile-time
3. it would only generate a warning, not an error
Cheers,
Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: "no variable or argument declarations are necessary."
Paul Rubin wrote:
>>Which is evaluated at runtime, does not require that the actual global
>>variable be pre-existing, and does not create the global variable if
>>not actually assigned. I think that is pretty different than your
>>proposal semantics.
>
>
> Different how?
Aren't you looking for some of compile-time checking that ensures that
only declared variables are actually used? If so, how does global help?
>>Your making this feature "optional" contradicts the subject of this
>>thread i.e. declarations being necessary.
>
> They're necessary if you enable the option.
OK. Would it work on a per-module basis or globally?
> def do_add(x->str, y->str):
> return '%s://%s' % (x, y)
>
> def do_something(node->Node):
> if node.namespace == XML_NAMESPACE:
> return do_add('http://', node.namespace)
> elif node.namespace == ...
Wouldn't an error be generated because XML_NAMESPACE is not declared?
And I notice that you are not doing any checking that "namespace" is a
valid attribute of the node object. Aren't the typos class of error that
you are looking to catch just as likely to occur for attributes as
variables?
Cheers,
Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: "no variable or argument declarations are necessary."
Paul Rubin wrote:
> You'd have to declare any variable global, or declare it local, or it
> could be a function name (defined with def) or a function arg (in the
> function scope), or maybe you could also declare things like loop
> indices. If it wasn't one of the above, the compiler would flag it.
OK. The Python compiler would check that the name is declared but it
would not check that it is defined before use? So this would be acceptable:
def foo():
local x
return x
>>OK. Would it work on a per-module basis or globally?
>
> Whatever perl does. I think that means per-module where the option is
> given as "use strict" inside the module.
>>>def do_add(x->str, y->str):
>>> return '%s://%s' % (x, y)
>>>def do_something(node->Node):
>>> if node.namespace == XML_NAMESPACE:
>>> return do_add('http://', node.namespace)
>>> elif node.namespace == ...
>>
>>Wouldn't an error be generated because XML_NAMESPACE is not declared?
>
>
> XML_NAMESPACE would be declared in the xml.dom module and the type
> info would carry over through the import.
Problems:
1. your type checking system is optional and xml.dom does not use it
1a. even if xml.dom did declare the type, what if the type were declared
conditionally e.g.
try:
unicode
except NameError:
XML_NAMESPACE = "..."
else:
XML_NAMESPACE = u"..."
2. the compiler does not have access to the names in other modules
anyway
>>And I notice that you are not doing any checking that "namespace" is a
>>valid attribute of the node object. Aren't the typos class of error
>>that you are looking to catch just as likely to occur for attributes
>>as variables?
>
>
> The node object is declared to be a Node instance and if the Node
> class definition declares a fixed list of slots, then the compiler
> would know the slot names and check them.
How would you find the class definition for the Node object at
compile-time? And by "slots" do you mean the existing Python slots
concept or something new?
> If the Node class doesn't
> declare fixed slots, then they're dynamic and are looked up at runtime
> in the usual way.
So only pre-defined slotted attributes would be accessable (if the
object uses slots). So the following would not work:
foo = Foo() # slots defined
foo.my_attribute = 'bar'
print foo.my_attribute
Cheers,
Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: "no variable or argument declarations are necessary."
Paul Rubin wrote: > Brian Quinlan <[EMAIL PROTECTED]> writes: > >>OK. The Python compiler would check that the name is declared but it >>would not check that it is defined before use? So this would be >>acceptable: >> >>def foo(): >> local x >> return x > > > Come on, you are asking silly questions. Any reasonable C compiler > would flag something like that and Python (with the flag set) should > do the same. If you want to ask substantive questions, that's fine, > but stop wasting our time with silly stuff. I'm not trying to be silly. I am trying to get a handle on the semantics that you are proposing. So we now have two requirements for the new declaration syntax (please let me know if I'm wrong): o the variable must be declared o the variable must be assigned I would assume that you would make it so that assignment and delaration happen as part of the same statement? > If type checking is implemented then the stdlib should be updated to > add declarations for public symbols. If not, the compiler would flag > the undeclared symbol. You could always declare it to be of type 'object'. Fair enough. > >> try: >> unicode >> except NameError: >> XML_NAMESPACE = "..." >> else: >> XML_NAMESPACE = u"..." > > > This wouldn't be allowed. OK, that sucks. > >>2. the compiler does not have access to the names in other modules anyway > > > You're being silly again. The compiler would examine the other module > when it processes the import statement, just like it does now. Right now, the compiler DOES NOT examine the contents of the other modules. All it does is generate an IMPORT_NAME instruction which is evaluation during runtime. So are you proposing that the compiler now scan other modules during compilation? >>How would you find the class definition for the Node object at >>compile-time? > > > By processing the xml.dom module when it's imported. Import happens at runtime (see above). But you seem to want compile-time type checking. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: "no variable or argument declarations are necessary."
Paul Rubin wrote:
>> Right now, the compiler DOES NOT examine the contents of the other
>> modules. All it does is generate an IMPORT_NAME instruction which is
>> evaluation during runtime.
>
>
>
> In that case the other module gets compiled when the IMPORT_NAME
> instruction is executed.
If compilation is required to complete the import (i.e. the module has
not already been imported and the module does not already have a
compiled byte-code file) then that is the same.
> That says that compile time and runtime are
> really the same thing in the current system.
I would say that they are separate systems but that the compilation
system is available to the runtime (but not vise-versa). In any case,
the important thing is that the operation of these systems has an impact
on the optional declaration proposal being discussed.
The thing that I'm trying to understand is how the proponents of such a
system would change Python to accomodate it. No one has been able to
articulate that with any decree of specifity ("Just do what Perl does").
Also, these (poorly-defined) proposals have revealed a lack of
understanding of Python's existing workings ("The compiler would examine
the other module when it processes the import statement, just like it
does now.").
Without a clear idea of the nature of the proposal, it is impossible to
assess it's costs and benefits. So could a proponent of optional
declarations please provide a more clear proposal?
Cheers,
Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: "no variable or argument declarations are necessary."
Paul Rubin wrote: > Brian Quinlan <[EMAIL PROTECTED]> writes: > >>Without a clear idea of the nature of the proposal, it is impossible >>to assess it's costs and benefits. So could a proponent of optional >>declarations please provide a more clear proposal? > > > There is no proposal on the table. There's a discussion of how this > stuff can work, and whether it's useful. As for how the compiler > deals with imported modules, see for example Common Lisp or Haskell or > ML--how do they do it? Except that you are providing neither useful input on how it could work nor on whether it would be useful. All you are doing is telling people to do research on other languages (that resumably do things in a way more to your liking). In conclusion, this thread is unlikely to make any useful progress. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Python Programming Contest
I've decided that it would be be fun to host a weekly Python programming contest. The focus will be on algorithms that require a bit of thought to design but not much code to implement. I'm doing to judge the solutions based on execution speed. It sucks but that is the easiest important consideration to objectively measure. I'll also indicated which solutions I think are good examples of Python design. Hopefully, people's solutions can provide a resource for people looking for best practice examples and also for people looking for performance ideas. You can find the first problem here: http://www.sweetapp.com/pycontest/contest1 I'm always looking for feedback, so let me know what you think or if you have any ideas for future problems. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Programming Contest
James wrote: > I am not sure if it is a good idea to use a LiveCD for OS when you are > testing for speed. CD access speeds fluctuate and may even impact > performance even if you start measuring after the module loading is > complete. It didn't seem to matter in my testing. Module loading is done before the test is run. Also, it is easiest to protect my system against malicious code if it is being run on an OS without a writeable filesystem. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Programming Contest
[EMAIL PROTECTED] wrote: > Brian> I've decided that it would be be fun to host a weekly Python > Brian> programming contest. The focus will be on algorithms that require > Brian> a bit of thought to design but not much code to implement. > > For some of us that's what we do day-in, day-out at work. It's just not > called a contest. To make it more challenging, we sometimes leave out the > "bit of thought" part. ;-) Hmmm...I find that I am rarely faced with challenging algorithmic problems in my day-to-day work. I continuously face difficult design decisions but that is a difficult sort of beast all together. This contest is for people who like thinking about algorithms. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Programming Contest
Bill Mill wrote: > On 7/15/05, Brian Quinlan <[EMAIL PROTECTED]> wrote: > >>[EMAIL PROTECTED] wrote: >> >>>Brian> I've decided that it would be be fun to host a weekly Python >>>Brian> programming contest. The focus will be on algorithms that require >>>Brian> a bit of thought to design but not much code to implement. >>> >>>For some of us that's what we do day-in, day-out at work. It's just not >>>called a contest. To make it more challenging, we sometimes leave out the >>>"bit of thought" part. ;-) >> >>Hmmm...I find that I am rarely faced with challenging algorithmic >>problems in my day-to-day work. I continuously face difficult design >>decisions but that is a difficult sort of beast all together. >> >>This contest is for people who like thinking about algorithms. >> >>Cheers, >>Brian >>-- >>http://mail.python.org/mailman/listinfo/python-list >> > > > Questions: > > Will that random test generator (included in the download) be used to > perform the actual testing? Yes. With two caveats: 1. I will pick the seed ahead of time so everyone gets the same problem set 2. I will use a local copy of the verification code (for performance reasons) > How many tests will be run on each > program? Probably a few thousand. If I need more to discriminate between two very similar solutions, then I will do so. > What is the penalty for a wrong answer? Infinite. Only correct solutions will be judged on performance. > PS - check out http://www.sleepinginairports.net/ before you say you > can't sleep in the airport :) Nice :-) Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Programming Contest
[EMAIL PROTECTED] wrote: > Brian> This contest is for people who like thinking about algorithms. > > Surely you must have missed the smiley... No, I saw it but it just confused me as I have no sense of humor. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Programming Contest
John Hazen wrote: > I have one question about the problem. Is the cost we are to minimize > the cost of arriving in the target city at all, or the cost of arriving > at the target city at the end of the final day of the schedule? Minimize the arrival cost. The arrival day is not relevant. > (If you were traveling to a conference, for example, you'd have a > specific arrival time, and a cost to stay in the destination city until > that time. But, if you were going to sight-see, then you could arrive > at any time, and begin your itinerary upon arrival.) Call it a vacation then :-) > Say I can find a combination of flights that gets me to the target at > the end of day 3 for 390 units, and a combination that gets me there at > the end of day 4 for 400. If you count the hostel cost from day 3 to > day 4, the first combination costs 410. So, which is preferred? The first option (day 3 for 390 units). > P.S. I just realized this may be answered be the test suite, but I'm > still at the thinking stage. No problem. Let me know if you have any other questions. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Programming Contest
ThanhNam Nguyen wrote: > Since my NNTP server doesnt allow posting, I'll ask you directly > instead. > > Must I start from the first day? No. > For example: > > 1st day: A --> B 100 bucks > 2nd day: A --> B 60 bucks > 3rd day: A --> B 40 bucks > > What would the solution be? And for how much in total? There are two correct solutions: ["A", "B"] # spend one night in A, then fly to B on day two (cost 80) ["A", "A", "B"] # spend two nights in A, then fly to B on day two (cost 80) Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Programming Contest
ThanhNam Nguyen wrote: >>>1st day: A --> B 100 bucks >>>2nd day: A --> B 60 bucks >>>3rd day: A --> B 40 bucks >>>What would the solution be? And for how much in total? >>> >> >>There are two correct solutions: >> >>["A", "B"] # spend one night in A, then fly to B on day two (cost 80) >>["A", "A", "B"] # spend two nights in A, then fly to B on day two >>(cost 80) > > > They all mean I must start from the first day. > > The best solution would be, I fly from A to B for 40 bucks on day 3, > assuming I live in the current city A. Then you should assume that you don't live in city A, because the actual cost in this case is 80. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Programming Contest
Raymond Hettinger wrote: > I'm curious about the stability of your timing setup. If you run your > own version of fly.py several times with the same starting seed, how > much variation do you see between runs? There is very little variation (about 0.1%) but my solution is over an order of magnitude slower than some of the submissions that I've gotten. It is likely that the overhead of my timing code is significant when running your solution. I may have to *slightly* revise my test code to get better results. I think that I can do so without changing the distribution of the random schedule so as not to be biased against some solutions. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
REMINDER: Python Contest
This is just a reminder that the deadline for my little programming competition is Monday. Original E-mail --- I've decided that it would be be fun to host a weekly Python programming contest. The focus will be on algorithms that require a bit of thought to design but not much code to implement. I'm doing to judge the solutions based on execution speed. It sucks but that is the easiest important consideration to objectively measure. I'll also indicated which solutions I think are good examples of Python design. Hopefully, people's solutions can provide a resource for people looking for best practice examples and also for people looking for performance ideas. You can find the first problem here: http://www.sweetapp.com/pycontest/contest1 I'm always looking for feedback, so let me know what you think or if you have any ideas for future problems. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Python Programming Contest: First results
Here are the results for the first problem in the Python Programming Contest. I haven't been able to find as much time as I excepted, so my analysis is not very in depth. You can find the results here: http://www.sweetapp.com/pycontest/contest1/results.html And the problem definition here: http://www.sweetapp.com/pycontest/contest1/ Kudos to everyone who participated but especially to Raymond Hettinger and Thomas Lotze, whose solutions were nearly 50 times faster than mine. I'd also like to point out that Thomas Guettler's solution, which is the slowest, was completed in less than 3 hours after the contest was announced. That's impresive for a correct solution. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Programming Contest: First results
Tomi Kyöstilä wrote: > Why don't I see my solution (__author__ = "dOb") in the results? I'm > sure that you got it as you replied to my mail. Ahhh...sorry. I have your solution and I timed it but I don't have the results here so I can't add it to the website. I'll do it tomorrow. > Where do the timing results come from? Is it the number that > fly_test.main(['fly']) outputs? I don't think it is that because with my > solution that would be ~0.06 seconds. This is the time required to do several thousand trials. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Programming Contest: First results
Brian Quinlan wrote: > Tomi Kyöstilä wrote: > >Why don't I see my solution (__author__ = "dOb") in the results? I'm >sure that you got it as you replied to my mail. Your solution is now included. See: http://www.sweetapp.com/pycontest/contest1/results.html Good job! Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Programming Contest: First results
Tomi Kyöstilä wrote: > Any idea when the next competition is coming? (it hasn't been quite > weekly as you hoped, eh? ;) Uh no. It turns out that I have less time than I thought, though a big chunk of it should be freed-up after this weekend. I do have an idea... :-) Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: What are modules really for?
N.Davis wrote: > Functions existing in a module? Surely if "everything is an object" > (OK thats Java-talk but supposedly Python will eventually follow this > too) There is a difference between everything being an object and everything being an instance of a class. In Python, every runtime entity is an object but not everything is a class instance. In Java, there are runtime-accessable entities that are neither objects nor class instances. > then there should be nothing in a module thats not part of a class. Even > a static method is simply a class function that operates on the > "collection of all instances" rather than a single instance. Really? What instances do the static methods in the Java Math class operate on? Not Math instances, of course. So what is the rational for them being packaged in the Math class? Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Writing a small battleship game server in Python
Michael Goettsche wrote:
> What would be a good, but still easy way to write such a server?
You could use SimpleXMLRPCServer. A client call sequence could like this:
>>> s = xmlrpclib.Server('http://...')
>>> token = s.join_game() # blocks until another player joins
>>> s.send_board(
... ['...pppa...',
... '..a...',
... '.da...',
... '.da.ss',
... '.da...'])
>>> s.get_other_board(token) # blocks until other player sends board
['...',
'...',
'...',
'...',
'...']
>>> s.fire(2,2)
'Hit!'
>>> s.get_other_board(token)
['...',
'.H.',
'...',
'...',
'...']
>>> s.get_my_board(token) # blocks until other guy shoots
['...pppa...',
'..a...',
'.da...',
'.dA.ss', # uppercase if hit
'.da...'])
Just designed in 2 minutes - but you get the idea.
Cheers,
Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: xmlrcp register classes
Sergio Rua wrote:
> Server = SimpleXMLRPCServer (('127.0.0.1',8080))
>
> Server.register_instance(MyClass1())
> Server.register_instance(MyClass2())
> Server.register_instance(MyClass3())
>
> What is seems to happen is that only the last class I register it is the
> only one being exported. How can I register all the classes? Thanks.
class MyCombinedClass(MyClass1, MyClass2, MyClass3):
pass
Server.register_instance(MyCombinedClass())
Cheers,
Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: First release of Shed Skin, a Python-to-C++ compiler.
Mark Dufour wrote: > The latter is certainly my goal. I just haven't looked into supporting > exceptions yet, because I personally never use them. I feel they > should only occur in very bad situations, or they become goto-like > constructs that intuitively feel very ugly. In the 5500 lines of the > compiler itself, I have not needed to use a single exception. For > example, I prefer to check whether a file exists myself, rather than > executing code that can suddenly jump somewhere else. There's probably > some use for exceptions, but I don't (want to?) see it. I don't understand your example here. When you check that a file exists, you feel safe that openning it will succeed? What if: o you don't have permission to open the file o the file is deleted in the time between you checking for it's existance and opening it (possible race condition) o the system doesn't have sufficient resources to open the file e.g. too many open file handles o the file is already open with exclusive read/write permission Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: First release of Shed Skin, a Python-to-C++ compiler.
Mark Dufour wrote: > You're right, I don't feel safe about that. It's a bad example. I just > prefer error codes, because the code usually becomes cleaner (at least > to me). I would really like it if I could do something like this: > > f = file(name) > if not f: > print 'error opening file %s: %s', name, str(f.error) > sys.exit() I think you meant: print 'error opening file %s: %s' % (name, str(f.error)) You forgot to check for an error when: o when you wrote f.error [attribute "error" might not exist e.g. f is None] o you called str(f.error) [might contain unicode characters that can't be converted to a string using the default encoding] o when you used the % operator [format string might be wrong] And, of course, pretty much every expression can result in a memory error. Exception handling is a boon because almost EVERY expression that you write can result in a error and checking each one is tedious in the extreme. So people forget and then their programs exhibit very odd behavior that is very difficult to debug. If you look at the Python C source, you'll notice that probably 50% of the code is devoted to error handling (that was a guess). Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this a bug? BOM decoded with UTF8
Diez B. Roggisch wrote: I know its easy (string.replace()) but why does UTF-16 do it on its own then? Is that according to Unicode standard or just Python convention? BOM is microsoft-proprietary crap. UTF-16 is defined in the unicode standard. What are you talking about? The BOM and UTF-16 go hand-and-hand. Without a Byte Order Mark, you can't unambiguosly determine whether big or little endian UTF-16 was used. If, for example, you came across a UTF-16 text file containing this hexidecimal data: 2200 what would you assume? That is is quote character in little-endian format or that it is a for-all symbol in big-endian format? For more details, see: http://www.unicode.org/faq/utf_bom.html#BOM Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this a bug? BOM decoded with UTF8
Diez B. Roggisch wrote:
I'm well aware of the need of a bom for fixed-size multibyte-characters like
utf16.
But I don't see the need for that on an utf-8 byte sequence, and I first
encountered that in MS tool output - can't remember when and what exactly
that was. And I have to confess that I attributed that as a stupidity from
MS. But according to the FAQ you mentioned, it is apparently legal in utf-8
too. Neverless the FAQ states:
[snipped]
So they admit that it makes no sense - especially as decoding a utf-8 string
given any 8-bit encoding like latin1 will succeed.
They say that it makes no sense as an byte-order indicator but they
indicate that it can be used as a file signature.
And I'm not sure what you mean about decoding a UTF-8 string given any
8-bit encoding. Of course the encoder must be know:
>>> u'T\N{LATIN SMALL LETTER U WITH DIAERESIS}r'
... .encode('utf-8').decode('latin1').encode('latin1')
'T\xc3\xbcr'
I can assume you that most Germans can differentiate between "Tür" and
"Tã¼r".
Using a BOM with UTF-8 makes it easy to indentify it as such AND it
shouldn't break any probably written Unicode-aware tools.
Cheers,
Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: best XSLT processor?
fanbanlo wrote: Which XSLT processor is the most reliable? requirement: + must support Python 2.4 + must run w/ Windows (and Linux) + not super slow My python project needs a XSLT processor + Docbook's XSLT to translate from docbook-xml -> HTML. PyXML? is it reliable? Slow? 4Suite? some said it is buggy (lots of work arounds)? Others ??? You could try Pyana (pyana.sourceforge.net). It's a binding to the Apache Group's Xalan engine. I don't think that it is a flexable as 4Suite but it is probably faster and more standards compliant (it was the last time that I checked). Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Small but significant memory leak in Pyana XSLT processor
Ola Natvig wrote: My problem: When serving my content without the XSLT processor the process size remains the same size. (about 18MB in the windows task manager). But when I use the XSLT processor the process will slowly gain size. About 1MB for each 10k requests, and this memory are not freed. I feel certain that the leak are located in the XSLT processor since the problem occurs when that is plugged into the system. There is not much wrapping code so I don't think the problem are of that manner. So I wonder if anyone have had the same experience with this XSLT toolkit before and if you were able to fix it. I'm the author of the Pyana package and, to the best of my knowledge, Pyana does not have any memory leaks. It seems like there are three possibilities: your code is leaking somehow (seems unlikely based on the above), Pyana is leaking (very possible) or Xalan is leaking (very possible). If Pyana is leaking then it is likely that the problem can be fairly easily isolated and fixed (by me). Would it be possible for you to isolate the leaking code and send it to me? If that is not possible, then let me know and I can recommend steps to help find the problem youself. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Small but significant memory leak in Pyana XSLT processor
Ola Natvig wrote: I've isolated the problem a bit more. The script will not leak if line 25: transformer.transform2String(transformer.parseSource(xml), xslt) is changed to: transformer.transform2String(xml, xslt) Looks like the DOM tree or at least parts of it are not freed. But there are no need to call parseSource when the XML is different for each iteration, that solves my problem, but the parseSoruce function should free all memory anyways. Yes it should. It's not obvious to me why it doesn't. I'll investage it in detail a bit later. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: IMAP UTF-7, any codec for that anywhere?
Max M wrote:
Is there any codec available for handling The special UTF-7 codec for IMAP?
I have searched the web for info, but there only seem to be discussions
about it. Not actual implementations.
Is there something special do you need or is recipe OK?
>>> u"\u00ff".encode('utf-7')
'+AP8-'
Cheers,
Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: mapping function to vars
[EMAIL PROTECTED] wrote: I need to map a function to several variables. I'm trying to use map and lambda to do this. Here's my attempt... #!/usr/bin/env python from random import * [fee, fye, foe, fum] = map(lambda n: random(), range(4)) from random import random fee = random() fye = random() foe = random(), fum = random() print fee print fye print foe print fum ...I'm essentially trying to map a function that takes no parameters to a group of variables. This works, but pychecker complains about the 'n' parameter. Is there a better way to do this? TIA -- http://mail.python.org/mailman/listinfo/python-list
Re: Python documentation moronicities (continued)
I think that there are some nice ideas in the new version e.g. "Regex functions" is a nicer title than "Module contents", examples, caveats. But there are some organizational problems and the actual writting is a bit weak. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Drawing a grid on a picture
Jive Dadson wrote: > I also found a reference to something called PIL. Maybe that's the > ticket. If so, where can I find it (with documentation)? Thanks. The will likely do what you want. And you can find it the same way that you would find anything online i.e. with google. But here is the link: http://www.pythonware.com/products/pil/ Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Syntax Highlighting Module
[EMAIL PROTECTED] wrote: > Hello, > I have an idea for a project which involves an editor that supports > syntax highlighting. This would be for any language, particularly php, > html, css, etc. I would like to write this program using python. It > would only make sense to base this upon existing open source code. My > question is there a python module or examples on how to write a code > editor which supports modulated syntax highlighting? Google: SilverCity Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Convert StringIO to string
Jonathan Bowlas wrote:
> Ahh thanks, I'll give that a try.
>
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of
> [EMAIL PROTECTED]
> Sent: 16 October 2006 14:00
> To: python-list@python.org
> Subject: Re: Convert StringIO to string
>
> Jonathan Bowlas wrote:
>> But obviously replace() isn't an attribute of StringIO so I guess I need
> to
>> convert it to a string first, can someone please advise how I can do this?
>
> StringIO objects are file-like objects, so you need to use read or
> readlines to get the string data out of it (just like a regular file).
> Before reading remember to seek back to the beginning to get all of the
> data ("Be kind, rewind!"):
>
import StringIO
s = StringIO.StringIO()
s.write("hello world\n")
s.seek(0)
s.read()
> 'hello world\n'
Or, instead of seak/read, just do
>>> s.getvalue()
Cheers,
Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: New to Python: Do we have the concept of Hash in Python?
A.M wrote:
> Is there any built-in Hash implementation in Python? I am looking for a
> container that I can access to it's items by name. Something like this:
>
> Print container["memeberName"]
d = {"memberName" : "some value"}
print d["memberName"]
> I am asking this because I learned that DB-API in Python doesn't offer
> access to cursor columns by name. The only option is access by index. I hope
> that I've got it wrong!
>
> I learned Ruby perfectly supports that.
Python does not use column names by default because they are not
portable across database implementations. That being said, you can get
them easily, if you want. Look at the "description" attribute of your
cursor instance.
Cheers,
Brian
--
http://mail.python.org/mailman/listinfo/python-list
Sampling a population
This is less a Python question and more a optimization/probability question. Imaging that you have a list of objects and there frequency in a population e.g. lst = [(a, 0.01), (b, 0.05), (c, 0.50), (d, 0.30), (e, 0.04), (f, 0.10)] and you want to drawn n items from that list (duplicates allowed), with that probability distribution. The fastest algorithm that I have been able to devise for doing so is: O(n * log(len(lst))). Can anyone think or a solution with a better time complexity? If not, is there an obviously better way to do this (assuming n is big and the list size is small). Here is the code: from random import random from bisect import bisect def draw(n, lst): ps = [] last = 0 for p in lst: ps.append(last + p) last += p # ps = [0.01, 0.06, 0.56, 0.86, 0.90, 1.00] chosen = [0] * len(lst) # track frequency for i in range(n): r = random() chosen[bisect(ps, r)] += 1 # binary search and increment result = [] # rescale probability based on frequency for c in chosen: result.append(float(c) / n) return result lst = [0.01, 0.05, 0.50, 0.30, 0.04, 0.10] print draw(1, lst) -- http://mail.python.org/mailman/listinfo/python-list
Re: Sampling a population
Robert Kern wrote: > [numpy implementation snipped] > Ed Schofield has an implementation of an algorithm by Marsaglia[1] which turns > sampling into a fast table lookup. If your probabilities have limited > precision > (2**-30 or so rather than the full double precision 2**-52 or so), then this > might be an attractive option. > > [1] http://www.jstatsoft.org/v11/i03/v11i03.pdf > Thanks a lot for the numpy implementation and for the literature reference! I'll try to figure out how little precision I need in my sampling. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Vancouver Python Workshop: New Keynoter
What's New? === The Vancouver Python Workshop is pleased to announce the addition of a third keynote speaker to this year's conference. Ian Cavén is the primary developer of the Lowry Digital Images motion picture restoration system. This Python and Zope-based system has been used to restore over 150 motion pictures. Highlights include Citizen Kane, Sunset Boulevard and both the Indiana Jones and Star Wars trilogies. While Ian was Chief Scientist at Lowry Digital, his rack of computers grew from a few Macintoshes on his desktop to over six hundred Macintosh and Linux servers - at one point earning Lowry the title as the second biggest installation of parallel processing Maintoshes in the world. In 2005, Lowry Digital Images was acquired by DTS (the famous movie audio company) and renamed DTS Digital Images. The motion picture restoration system has been discussed in publications as diverse as IEEE Spectrum, USA Today, the BBC NEWS, the New York Times and Apple.com. Ian has been a Python enthusiast since 1999. About the Vancouver Python Workshop === The conference will begin with keynote addresses on August 4st by Guido van Rossum [1], Jim Hugunin [2], and Ian Cavén. Further talks (and tutorials for beginners) will take place on August 5th and 6th. The Vancouver Python Workshop is a community organized conference designed for both the beginner and for the experienced Python programmer with: * tutorials for beginning programmers * advanced lectures for Python experts * case studies of Python in action * after-hours social events * informative keynote speakers * tracks on multimedia, Web development, education and more More information see: http://www.vanpyz.org/conference/ or contact Brian Quinlan at: [EMAIL PROTECTED] Vancouver = In addition to the opportunity to learn and socialize with fellow Pythonistas, the Vancouver Python Workshop also gives visitors the opportunity to visit one of the most extraordinary cities in the world [3]. For more information about traveling to Vancouver, see: http://www.vanpyz.org/conference/vancouver.html http://www.tourismvancouver.com http://en.wikipedia.org/wiki/Vancouver Important dates === Talk proposals accepted: May 15th to June 15th Early registration (discounted): May 22nd to June 30th Normal registration: from July 1st Keynotes: August 4th Conference and tutorial dates: August 5th and 6th [1] Guido van Rossum (Google) is the inventor of Python and has managed its growth and development for more than a decade. Guido was awarded the Free Software Foundation Award in 2002 and Dr.Dobb's 1999 Excellence in Programming Award. Guido works at Google and spends half of his time on Python. [2] Jim Hugunin (Microsoft) is the creator of numerous innovations that take Python into new application domains. Jim's most recent project, IronPython integrates Python into Microsoft's .NET runtime. Jim's previous project, Jython is Python for the Java runtime and was the second production-quality implementation of Python. Before that, Jim's Numeric Python adapted Python to the needs of number crunching applications. Jim works at Microsoft adapting the .NET runtime to the needs of dynamic languages like Python. [3] http://news.bbc.co.uk/2/hi/business/2299119.stm Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Vancouver Python Workshop - Talk submission reminder
What's New? === The deadline for submitting a talk or tutorial for the Vancouver Python Workshop is fast approaching. Talks will be accepted until Friday June 16th. To submit a talk, see: http://www.vanpyz.org/conference/talksubmission.html About the Vancouver Python Workshop === The conference will begin with keynote addresses on August 4st by Guido van Rossum [1], Jim Hugunin [2], and Ian Cavén [3]. Further talks (and tutorials for beginners) will take place on August 5th and 6th. The Vancouver Python Workshop is a community organized and designed for both the beginner and for the experienced Python programmer with: * tutorials for beginning programmers * advanced lectures for Python experts * case studies of Python in action * after-hours social events * informative keynote speakers * tracks on multimedia, Web development, education and more More information see: http://www.vanpyz.org/conference/ or contact Brian Quinlan at: [EMAIL PROTECTED] Vancouver = In addition to the opportunity to learn and socialize with fellow Pythonistas, the Vancouver Python Workshop also gives visitors the opportunity to visit one of the most extraordinary cities in the world [4]. For more information about traveling to Vancouver, see: http://www.vanpyz.org/conference/vancouver.html http://www.tourismvancouver.com http://en.wikipedia.org/wiki/Vancouver Important dates === Talk proposals accepted: May 15th to June 15th Early registration (discounted): May 22nd to June 30th Normal registration: from July 1st Keynotes: August 4th Conference and tutorial dates: August 5th and 6th [1] Guido van Rossum (Google) is the inventor of Python and has managed its growth and development for more than a decade. Guido was awarded the Free Software Foundation Award in 2002 and Dr.Dobb's 1999 Excellence in Programming Award. Guido works at Google and spends half of his time on Python. [2] Jim Hugunin (Microsoft) is the creator of numerous innovations that take Python into new application domains. Jim's most recent project, IronPython integrates Python into Microsoft's .NET runtime. Jim's previous project, Jython is Python for the Java runtime and was the second production-quality implementation of Python. Before that, Jim's Numeric Python adapted Python to the needs of number crunching applications. Jim works at Microsoft adapting the .NET runtime to the needs of dynamic languages like Python. [3] Ian Cavén is the primary developer of the Lowry Digital Images motion picture restoration system. This Python and Zope-based system has been used to restore over 150 motion pictures. Highlights include Citizen Kane, Sunset Boulevard and both the Indiana Jones and Star Wars trilogies. While Ian was Chief Scientist at Lowry Digital, his rack of computers grew from a few Macintoshes on his desktop to over six hundred Macintosh and Linux servers - at one point earning Lowry the title as the second biggest installation of parallel processing Maintoshes in the world. In 2005, Lowry Digital Images was acquired by DTS (the famous movie audio company) and renamed DTS Digital Images. The motion picture restoration system has been discussed in publications as diverse as IEEE Spectrum, USA Today, the BBC NEWS, the New York Times and Apple.com. Ian has been a Python enthusiast since 1999. [4] http://news.bbc.co.uk/2/hi/business/2299119.stm Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Possible inaccuracy in Python 2.4 when using profiler calibration
I have a misinformed theory that I'd like to share with the list.
I believe that profiler calibration no longer makes sense in Python 2.4
because C functions are tracked and they have a different call overhead
than Python functions (and calibration is done only using Python
functions). Here is my reasoning (in code form):
% c:\python24\python
...
>>> import profile
>>> p = profile.Profile()
>>> for i in range(5):
... print p.calibrate(100)
...
4.00375499355e-006
3.95700929469e-006
3.94034406478e-006
4.00315854962e-006
3.99454335716e-006
Code
import profile
profile.Profile.bias = 3.90e-006 # very conservative
def bar():
l = []
for i in range(10):
l += [i]
def foo():
l = []
for i in range(10):
l.append(i) # C function that can be tracked
def baz():
bar()
foo()
profile.run('baz()', "out.prof")
from pstats import Stats
s = Stats('out.prof')
s.sort_stats('time', 'calls')
s.print_stats()
from timeit import Timer
t1 = Timer(
'bar()', 'from __main__ import bar',)
print 'bar():', t1.timeit(1000) / 1000
t2 = Timer(
'foo()', 'from __main__ import foo',)
print 'foo():', t2.timeit(1000) / 1000
Output
--
Thu Jun 15 10:22:29 2006out.prof
18 function calls in -0.090 CPU seconds
Ordered by: internal time, call count
ncalls tottime percall cumtime percall filename:lineno(function)
10.0580.0580.0620.062 cal-test.py:4(bar)
20.0060.0030.0060.003 :0(range)
10.0040.004 -0.090 -0.090 cal-test.py:14(baz)
10.0010.0010.0010.001 :0(setprofile)
10.0000.000 -0.090 -0.090 profile:0(baz())
10.0000.000 -0.090 -0.090 :1(?)
00.000 0.000 profile:0(profiler)
1 -0.066 -0.066 -0.157 -0.157 cal-test.py:9(foo)
10 -0.094 -0.000 -0.094 -0.000 :0(append)
bar(): 0.0582713573932
foo(): 0.0370039776005
Analysis
As you can see, the profiler result for "bar" is pretty reasonable but
it is not for "foo" or "append". I believe that is because the calling
of the C function "append" takes less time than is accounted for in the
bias measurement (which was generated by measuring the call time of a
Python function).
So the bias computation doesn't make sense in Python 2.4.
What do y'all think? Is this a well known fact? Should I construct a
test to see if C function call overhead is actually less than Python
function call overhead?
Cheers,
Brian
--
http://mail.python.org/mailman/listinfo/python-list
Vancouver Python Workshop - Talk submission deadline
What's New? === This is your last change to submit a talk for the Vancouver Python Workshop. Talks will be accepted until Friday June 16th. This is a great opportunity for you to share your project or interests with the Python community, so please take advantage of it! To submit a talk, see: http://www.vanpyz.org/conference/talksubmission.html About the Vancouver Python Workshop === The conference will begin with keynote addresses on August 4st by Guido van Rossum [1], Jim Hugunin [2], and Ian Cavén [3]. Further talks (and tutorials for beginners) will take place on August 5th and 6th. The Vancouver Python Workshop is a community organized and designed for both the beginner and for the experienced Python programmer with: * tutorials for beginning programmers * advanced lectures for Python experts * case studies of Python in action * after-hours social events * informative keynote speakers * tracks on multimedia, Web development, education and more More information see: http://www.vanpyz.org/conference/ or contact Brian Quinlan at: [EMAIL PROTECTED] Vancouver = In addition to the opportunity to learn and socialize with fellow Pythonistas, the Vancouver Python Workshop also gives visitors the opportunity to visit one of the most extraordinary cities in the world [4]. For more information about traveling to Vancouver, see: http://www.vanpyz.org/conference/vancouver.html http://www.tourismvancouver.com http://en.wikipedia.org/wiki/Vancouver Important dates === Talk proposals accepted: May 15th to June 15th Early registration (discounted): May 22nd to June 30th Normal registration: from July 1st Keynotes: August 4th Conference and tutorial dates: August 5th and 6th [1] Guido van Rossum (Google) is the inventor of Python and has managed its growth and development for more than a decade. Guido was awarded the Free Software Foundation Award in 2002 and Dr.Dobb's 1999 Excellence in Programming Award. Guido works at Google and spends half of his time on Python. [2] Jim Hugunin (Microsoft) is the creator of numerous innovations that take Python into new application domains. Jim's most recent project, IronPython integrates Python into Microsoft's .NET runtime. Jim's previous project, Jython is Python for the Java runtime and was the second production-quality implementation of Python. Before that, Jim's Numeric Python adapted Python to the needs of number crunching applications. Jim works at Microsoft adapting the .NET runtime to the needs of dynamic languages like Python. [3] Ian Cavén is the primary developer of the Lowry Digital Images motion picture restoration system. This Python and Zope-based system has been used to restore over 150 motion pictures. Highlights include Citizen Kane, Sunset Boulevard and both the Indiana Jones and Star Wars trilogies. While Ian was Chief Scientist at Lowry Digital, his rack of computers grew from a few Macintoshes on his desktop to over six hundred Macintosh and Linux servers - at one point earning Lowry the title as the second biggest installation of parallel processing Maintoshes in the world. In 2005, Lowry Digital Images was acquired by DTS (the famous movie audio company) and renamed DTS Digital Images. The motion picture restoration system has been discussed in publications as diverse as IEEE Spectrum, USA Today, the BBC NEWS, the New York Times and Apple.com. Ian has been a Python enthusiast since 1999. [4] http://news.bbc.co.uk/2/hi/business/2299119.stm Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Vancouver Python Conference: T-Shirt design contest
The Vancouver Python Workshop organizers are having problems coming up with text for our T-Shirts (don't worry: we already have the graphics figured out). We want something that matches Python's simplicity and elegance. So we're asking for your help. If you submit the text that we end up using, I'll send you 3 T-Shirts (1 seems too cheap, and there is always wear-and-tear, one for your cat, etc.). If we have extra T-Shirts, we might also send them to other people who had good (but not winning) ideas. So please go here and make a suggestion: http://www.vanpyz.org/conference/tshirt_contest.html BTW, you don't have to attend the conference to participate in the T-Shirt design. But you should attend the conference anyway, because it is going to be great. For information on the workshop, see: http://www.vanpyz.org/conference/ Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: VPW: T-Shirt design contest
Bruno Desthuilliers submitted this really cool rant/essay/something from Tim Lesher that I hadn't seen before. I think that the original source is: http://apipes.blogspot.com/2005/01/choose-python.html Choose Python. Choose readability. Choose the simple over the complex and the complex over the complicated. Choose dynamic typing. Choose duck typing. Choose decorators. Choose generators. Choose metaclasses if you don’t value your sanity. Choose to import this. Choose an almost-fanatical devotion to the BDFL, unless he comes up with something like optional static typing, in which case choose to whine about it in your blog until he stops. Choose Effbot. Choose Timbot. Choose wx. Choose to come up with a bloody implementation before spouting off on comp.lang.python or Python-Dev. Choose the explicit rather than the implicit. Choose one obvious way to do it, especially if you are Dutch. Choose list comprehensions. Choose Paul Graham’s essays and s/LISP/Python/g. Choose Jython when your marketing people choose Java. Choose speed of development over speed of execution, but when in doubt, import psyco. Choose to finish early and laugh at your colleagues as they waste their miserable lives bowing down in subservience to that sadistic little C++ compiler. Choose your future. Choose Python. I think that it might be a bit long to put on a T-Shirt but it is definitely cool :-) Still collecting ideas at: http://www.vanpyz.org/conference/tshirt_contest.html Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
VPW: Talk schedule available
What's New? === The talk schedule for the Vancouver Python Workshop is now available: http://www.vanpyz.org/conference/talkschedule.html This years line-up might be even stronger than in 2004, so check it out! About the Vancouver Python Workshop === The conference will begin with keynote addresses on August 4st by Guido van Rossum [1] and Jim Hugunin [2]. Further talks (and tutorials for beginners) will take place on August 5th and 6th. Ian Cavén [3] will give the closing address. The Vancouver Python Workshop is a community organized and designed for both the beginner and for the experienced Python programmer with: * tutorials for beginning programmers * advanced lectures for Python experts * case studies of Python in action * after-hours social events * informative keynote speakers * tracks on multimedia, Web development, education and more More information see: http://www.vanpyz.org/conference/ or contact Brian Quinlan at: [EMAIL PROTECTED] Vancouver = In addition to the opportunity to learn and socialize with fellow Pythonistas, the Vancouver Python Workshop also gives visitors the opportunity to visit one of the most extraordinary cities in the world [4]. For more information about traveling to Vancouver, see: http://www.vanpyz.org/conference/vancouver.html http://www.tourismvancouver.com http://en.wikipedia.org/wiki/Vancouver Important dates === Talk proposals accepted: May 15th to June 15th Early registration (discounted): May 22nd to June 30th Normal registration: from July 1st Keynotes: August 4th Conference and tutorial dates: August 5th and 6th [1] Guido van Rossum (Google) is the inventor of Python and has managed its growth and development for more than a decade. Guido was awarded the Free Software Foundation Award in 2002 and Dr.Dobb's 1999 Excellence in Programming Award. Guido works at Google and spends half of his time on Python. [2] Jim Hugunin (Microsoft) is the creator of numerous innovations that take Python into new application domains. Jim's most recent project, IronPython integrates Python into Microsoft's .NET runtime. Jim's previous project, Jython is Python for the Java runtime and was the second production-quality implementation of Python. Before that, Jim's Numeric Python adapted Python to the needs of number crunching applications. Jim works at Microsoft adapting the .NET runtime to the needs of dynamic languages like Python. [3] Ian Cavén is the primary developer of the Lowry Digital Images motion picture restoration system. This Python and Zope-based system has been used to restore over 150 motion pictures. Highlights include Citizen Kane, Sunset Boulevard and both the Indiana Jones and Star Wars trilogies. While Ian was Chief Scientist at Lowry Digital, his rack of computers grew from a few Macintoshes on his desktop to over six hundred Macintosh and Linux servers - at one point earning Lowry the title as the second biggest installation of parallel processing Maintoshes in the world. In 2005, Lowry Digital Images was acquired by DTS (the famous movie audio company) and renamed DTS Digital Images. The motion picture restoration system has been discussed in publications as diverse as IEEE Spectrum, USA Today, the BBC NEWS, the New York Times and Apple.com. Ian has been a Python enthusiast since 1999. [4] http://news.bbc.co.uk/2/hi/business/2299119.stm http://edition.cnn.com/2006/BUSINESS/06/15/btn.cities/index.html Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
VPW: early registration deadline
What's New? === The deadline for early-bird registration for the Vancouver Python Workshop is this Friday, June 30th. Early-bird registration is significantly discounted over normal registration, so register now at: http://www.vanpyz.org/conference/registration.html About the Vancouver Python Workshop === The conference will begin with keynote addresses on August 4st by Guido van Rossum [1] and Jim Hugunin [2]. Further talks (and tutorials for beginners) will take place on August 5th and 6th. Ian Cavén [3] will give the closing address. The Vancouver Python Workshop is a community organized and designed for both the beginner and for the experienced Python programmer with: * tutorials for beginning programmers * advanced lectures for Python experts * case studies of Python in action * after-hours social events * informative keynote speakers More information see: http://www.vanpyz.org/conference/ or contact Brian Quinlan at: [EMAIL PROTECTED] Vancouver = In addition to the opportunity to learn and socialize with fellow Pythonistas, the Vancouver Python Workshop also gives visitors the opportunity to visit one of the most extraordinary cities in the world [4]. For more information about traveling to Vancouver, see: http://www.vanpyz.org/conference/vancouver.html http://www.tourismvancouver.com http://en.wikipedia.org/wiki/Vancouver Important dates === Early registration (discounted): May 22nd to June 30th Normal registration: from July 1st Keynotes: August 4th Conference and tutorial dates: August 5th and 6th [1] Guido van Rossum (Google) is the inventor of Python and has managed its growth and development for more than a decade. Guido was awarded the Free Software Foundation Award in 2002 and Dr.Dobb's 1999 Excellence in Programming Award. Guido works at Google and spends half of his time on Python. [2] Jim Hugunin (Microsoft) is the creator of numerous innovations that take Python into new application domains. Jim's most recent project, IronPython integrates Python into Microsoft's .NET runtime. Jim's previous project, Jython is Python for the Java runtime and was the second production-quality implementation of Python. Before that, Jim's Numeric Python adapted Python to the needs of number crunching applications. Jim works at Microsoft adapting the .NET runtime to the needs of dynamic languages like Python. [3] Ian Cavén is the primary developer of the Lowry Digital Images motion picture restoration system. This Python and Zope-based system has been used to restore over 150 motion pictures. Highlights include Citizen Kane, Sunset Boulevard and both the Indiana Jones and Star Wars trilogies. While Ian was Chief Scientist at Lowry Digital, his rack of computers grew from a few Macintoshes on his desktop to over six hundred Macintosh and Linux servers - at one point earning Lowry the title as the second biggest installation of parallel processing Maintoshes in the world. In 2005, Lowry Digital Images was acquired by DTS (the famous movie audio company) and renamed DTS Digital Images. The motion picture restoration system has been discussed in publications as diverse as IEEE Spectrum, USA Today, the BBC NEWS, the New York Times and Apple.com. Ian has been a Python enthusiast since 1999. [4] http://news.bbc.co.uk/2/hi/business/2299119.stm http://edition.cnn.com/2006/BUSINESS/06/15/btn.cities/index.html Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
VPW: early registration deadline today!
What's New? === The deadline for early-bird registration for the Vancouver Python Workshop is today! Early-bird registration is significantly discounted over normal registration, so register now at: http://www.vanpyz.org/conference/registration.html About the Vancouver Python Workshop === The conference will begin with keynote addresses on August 4st by Guido van Rossum [1] and Jim Hugunin [2]. Further talks (and tutorials for beginners) will take place on August 5th and 6th. Ian Cavén [3] will give the closing address. The Vancouver Python Workshop is a community organized and designed for both the beginner and for the experienced Python programmer with: * tutorials for beginning programmers * advanced lectures for Python experts * case studies of Python in action * after-hours social events * informative keynote speakers More information see: http://www.vanpyz.org/conference/ or contact Brian Quinlan at: [EMAIL PROTECTED] Vancouver = In addition to the opportunity to learn and socialize with fellow Pythonistas, the Vancouver Python Workshop also gives visitors the opportunity to visit one of the most extraordinary cities in the world [4]. For more information about traveling to Vancouver, see: http://www.vanpyz.org/conference/vancouver.html http://www.tourismvancouver.com http://en.wikipedia.org/wiki/Vancouver Important dates === Early registration (discounted): May 22nd to June 30th Normal registration: from July 1st Keynotes: August 4th Conference and tutorial dates: August 5th and 6th [1] Guido van Rossum (Google) is the inventor of Python and has managed its growth and development for more than a decade. Guido was awarded the Free Software Foundation Award in 2002 and Dr.Dobb's 1999 Excellence in Programming Award. Guido works at Google and spends half of his time on Python. [2] Jim Hugunin (Microsoft) is the creator of numerous innovations that take Python into new application domains. Jim's most recent project, IronPython integrates Python into Microsoft's .NET runtime. Jim's previous project, Jython is Python for the Java runtime and was the second production-quality implementation of Python. Before that, Jim's Numeric Python adapted Python to the needs of number crunching applications. Jim works at Microsoft adapting the .NET runtime to the needs of dynamic languages like Python. [3] Ian Cavén is the primary developer of the Lowry Digital Images motion picture restoration system. This Python and Zope-based system has been used to restore over 150 motion pictures. Highlights include Citizen Kane, Sunset Boulevard and both the Indiana Jones and Star Wars trilogies. While Ian was Chief Scientist at Lowry Digital, his rack of computers grew from a few Macintoshes on his desktop to over six hundred Macintosh and Linux servers - at one point earning Lowry the title as the second biggest installation of parallel processing Maintoshes in the world. In 2005, Lowry Digital Images was acquired by DTS (the famous movie audio company) and renamed DTS Digital Images. The motion picture restoration system has been discussed in publications as diverse as IEEE Spectrum, USA Today, the BBC NEWS, the New York Times and Apple.com. Ian has been a Python enthusiast since 1999. [4] http://news.bbc.co.uk/2/hi/business/2299119.stm http://edition.cnn.com/2006/BUSINESS/06/15/btn.cities/index.html Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Behaviour of classes (tired of writing too much)
[EMAIL PROTECTED] wrote: > How do I catch any reference to an instance of a class, i.e., I want to > run some code as soon as an instance of a class is used in any way. > (and I don't want to define all of __add__, __ge__ etc etc etc etc etc) What do you mean by "used in any way"? Which of these are considered usage: >>> a = Foo() # obviously >>> b = a # ? >>> a.__gt__(5) # apparently >>> a.__gt__# ? Anyway, look into __getattr__ Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: A critique of cgi.escape
Jon Ribbens wrote: > In article <[EMAIL PROTECTED]>, Fredrik Lundh wrote: >>> It's up to me to decide whether or not an argument is good enough to >>> convince me, thank you very much. >> not if you expect anyone to take anything you say seriously. > > Now you're just being ridiculous. In this thread you have been rude, > evasive, insulting, vague, hypocritical, and have failed to answer > substantive points in favour of sarcastic and erroneous sniping - I'd > suggest it's you that needs to worry about being taken seriously. Actually, at least in the context of this mailing list, Fredrik doesn't have to worry about that at all. Why? Because he is one of the most prolific contributers to the Python language and libraries and his contributions have been of consistent high quality. You, on the other hand, are "just some guy" and people don't have a lot of incentive to convince you of anything. I have no opinion on the actual debate though. Just trying to help with the social analysis :-) Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: A critique of cgi.escape
Paul Rubin wrote: > Brian Quinlan <[EMAIL PROTECTED]> writes: >> o cgi.escape is not meant for serious web application development, > > What is it meant for then? Why should the library ever implement > anything in a half-assed way unsuitable for serious application > development, if it can supply a robust implementation instead? I'd have to dig through the revision history to be sure, but I imagine that cgi.escape was originally only used in the cgi module (and there only in it's various print_* functions). Then it started being used by other core Python modules e.g. cgitb, DocXMLRPCServer. The "mistake", if there was one, was probably that escape wasn't spelled _escape and got documented in the LaTeX documentation system. All of this is just speculation though. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: A critique of cgi.escape
Jon Ribbens wrote: > In article <[EMAIL PROTECTED]>, Brian Quinlan wrote: >> A summary of this pointless argument: > > Your summary seems pretty reasonable, but please note that later on, > the thread was not about cgi.escape escaping (or not) quote > characters (as described in your summary), but about Fredrik arguing, > somewhat incoherently, that it should have to take character encodings > into consideration. And, of course, about you telling people that their explanations are not good enough :-) BTW, I am curious about how you do unit testing. The example that I used in my summary is a very common pattern but would break in cgi.escape changed it's semantics. What do you do instead? Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: A critique of cgi.escape
A summary of this pointless argument:
Why cgi.escape should be changed to escape double quote (and maybe
single quote) characters by default:
o escaping should be very aggressive by default to avoid subtle bugs
o over-escaping is not likely to harm most program significantly
o people who do not read the documentation may be surprised by it's
behavior
Why cgi.escape should NOT be changed:
o it is current used in lots of code and changing it will almost
certainly break some of it, test suites at minimum e.g.
assert my_template_system("{foo}", foo='"') == '"'
o escaping attribute values is less common than escaping element
text so people should not be punished with:
- harder to read output
- (slightly) increased file size
- (slightly) decreased performance
o cgi.escape is not meant for serious web application development, so
either roll your own (trivial) function to do escaping how you want
it or use the one provided by your framework (if it is not automatic)
o the documentation describes the current behavior precisely and
suggests solutions that provide more aggressive escaping, so arguing
about surprising behavior is not reasonable
o it doesn't even make sense for an escape function to exist in the cgi
module, so it should only be used by old applications for
compatibility reasons
Cheers,
Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: A critique of cgi.escape
Jon Ribbens wrote:
> I guess, if you mean the part of the thread which went "it'll break
> existing code", "what existing code"? "existing code" "but what
> existing code?" "i dunno, just, er, code" "ok *how* will it break it?"
> "i dunno, it just will"?
See below for a possible example.
>> BTW, I am curious about how you do unit testing. The example that I used
>> in my summary is a very common pattern but would break in cgi.escape
>> changed it's semantics. What do you do instead?
>
> To be honest I'm not sure what *sort* of code people test this way. It
> just doesn't seem appropriate at all for web page generating code.
Well, there are dozens (hundreds?) of templating systems for Python.
Here is a (simplified/modified) unit test for my company's system (yeah,
we lifted some ideas from Django):
test.html
-
{foo | escape}
test.py
---
t = Template("test.html")
t['foo'] = 'Brian -> "Hi!"'
assert str(t) == 'Brian -> "Hi"'
So how would you test our template system?
> Web
> pages need to be manually viewed in web browsers, and validated, and
> checked for accessibility.
True.
> Checking they're equal to a particular
> string just seems bizarre (and where does that string come from
> anyway?)
Maybe, which is why I'm asking you how you do it. Some of our web
applications contain 100s of script generated pages. Testing each one by
hand after making a change would be completely impossible. So we use
HTTP scripting for testing purposes i.e. send this request, grab the
results, verify that the test in the element with id="username" equals
"Brian Quinlan", etc. The test also validates that each page is well
formed. We also view each page at some point but not every time a
developer makes a change that might (i.e. everything) affect the entire
system.
Cheers,
Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: A critique of cgi.escape
Jon Ribbens wrote:
> In article <[EMAIL PROTECTED]>, Brian Quinlan wrote:
>> Well, there are dozens (hundreds?) of templating systems for Python.
>
> I know, I wrote one of them ;-)
>
>> t = Template("test.html")
>> t['foo'] = 'Brian -> "Hi!"'
>> assert str(t) == 'Brian -> "Hi"'
>>
>> So how would you test our template system?
>
> What I don't get is why you are testing the above code like that at
> all. Surely if the template system somehow became so broken that it
> couldn't even do trivial replacements, you would notice immediately
> as all your web pages would go totally wrong.
If, in the example that I showed, the less-than character was not
correctly escaped, then it might not manifest itself frequently in a
typical application because the less-than character is seldom used in
English prose.
Also, assuming that single case was trivial to test without a test
harness, how many web pages do I have to look at to be reasonably
confident that *every* feature works correctly?
Cheers,
Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: A critique of cgi.escape
John Bokma wrote:
>> Why cgi.escape should NOT be changed:
>> o it is current used in lots of code and changing it will almost
>>certainly break some of it, test suites at minimum e.g.
>>assert my_template_system("{foo}", foo='"') == '"'
>
> You must be kidding.
Nope. How do you write your templating system unit tests?
>> o escaping attribute values is less common than escaping element
>>text
>
> Again, you must be kidding: href="/search.cgi?query=3&results=10"
>
Actually, I wasn't kidding. I was basing this belief on greping through
the Python standard library where only the quote=None form is ever used.
It also matches my experience. But I don't have a large enough sample to
make any claim either way.
Cheers,
Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: len(var) is [CONSTANT] equal to len(var) == [CONSTANT]?
Fredrik Lundh wrote:
> 4) [] and {} always create a new object every time they're evaluated.
Not quite. The empty tuple is cached:
>>> a = ()
>>> b = ()
>>> a is b
True
Cheers,
Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: len(var) is [CONSTANT] equal to len(var) == [CONSTANT]?
Fredrik Lundh wrote:
> Brian Quinlan wrote:
>
>>> 4) [] and {} always create a new object every time they're evaluated.
>> Not quite. The empty tuple is cached:
>>
>> >>> a = ()
>> >>> b = ()
>> >>> a is b
>> True
>
> () isn't [] or {}, though. time to switch to a bigger font? ;-)
Yeah, sorry I'm an idiot. I can't believe that I've been able to program
successfully for so long when I can't recognize the different between a
dictionary and a tuple :-)
Cheers,
Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: Advice needed on __del__
phil wrote: > I'm probably dense and don't quite get the point, even though I > have coded about 200,000 lines of Python and coded .5 million lines of > C before there was a C++ > > A class instance based language doesn't have to BE C++ to have a > destructor method. Very true. But meaningful finalizers are a difficult problem in languages like Python. For example: class A: def __del__(self): print self.b class B: def __init__(self, a): self.a = a def __del__(self): print self.a a = A() a.b = B(a) del a What should happen? How would you implement that behavior? How would you implement that behavior on Java and .NET (neither of which gaurantee that finalizers/destructors will be called). > All I wanted to know is: is the apparent treatment of __del__ > which SEEMS to work fine for me in 2.3.4 deprecated or mangled > in some later release or future plans. __del__ is not currently gauranteed to do anything. Currently it does something. Depending on that something is a bad idea. Could you explain your problem in more detail - maybe we'll have some suggestions. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Interactive shell for demonstration purposes
Can anyone recommend a Python interactive shell for use in presentations? Ideal characteristics (priority order): o configurable font size o full screen mode o readline support o syntax coloring I've tried ipython but, since it runs inside a console window, and the console window has a limited number of selectable fonts, it doesn't work terribly well. I've seen presentations using some sort of PyGame implemented shell. Does anyone have an information on that? Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Interactive shell for demonstration purposes
Ville Vainio wrote: > Hmm, do you consider the fonts in a console window unreadable? In fullscreen mode, yes (you get no choice of font size in Windows XP). In Windowed mode you still only get a limited font selection (only two fonts and only a few type sizes [most of which are small]). > I've > given a few presentations using ipython on win32 and it worked alright > - but I suppose the projector quality is a factor here... I'll get by but I was hoping for something better. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Interactive shell for demonstration purposes
Bengt Richter wrote: > If you make your console 96 wide and set the font to Lucida Console Bold > 24point, > it will probably expand to near full screen on 1024x768. You can set the > scroll buffer > to a couple hundred lines and adjust console widow height to suit. Use the > properties > from the system icon, or get there by Alt-Space P etc. That's not bad. There are two caveats: 1. you have to set the width to 72 characters (instead of 80) at 1024x768 with 24 point fonts 2. you can't run in full-screen mode Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Building a Python app with Mozilla
Kevin Walzer wrote: > Komodo is not a Python application. It is a Mozilla application that > supports Python development. Komodo is more akin to Thunderbird and > Firefox than anything else; it uses the XUL framework for rendering > widgets, if I'm not mistaken. If you want to build an application like > Komodo, get a book on developing with the Mozilla framework (XUL, XPCOM, > and all that) and look at that. Python has little to do with that. Most application logic in Komodo is implemented in Python, using the PyXPCOM bindings. The UI is implemented using XUL and JavaScript. The editor is Scintilla (C++). ../Komodo Edit.app/Contents/MacOS % find . -name "*.py" | xargs wc ... ... 126392 456858 4949602 total This doesn't include the python code in the Python libraries themselves. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: XMLRPC Server
[EMAIL PROTECTED] wrote: > Hi, I'm trying to create an XMLRPC server using apache + python (cgi). > It's not too difficult to configure everything, but I would like to > tune it in order to receive up to 2000 calls per minute without any > problems. That doesn't seem like excessive volume. Why not just try it? You could replace your database logic with time.sleep(1) for now. > Do Pthon CGIs use threading? To do what? CGI requires that a new interpreter instance be launched to handle every request. The requests will be handled in parallel with the number of requests handled simultaneously depending on your apache configuration. > I need to make it very efficient, Actually, you might not have to. 2000 calls/minute isn't that big, assuming you have a decent server. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: XMLRPC Server
Fredrik Lundh wrote: > well, if you're talking pure CGI, you need to start the interpreter, > import the required modules, connect to the database, unmarshal the > xml-rpc request, talk to the database, marshal the response, and shut > down, in less than 30 milliseconds. > > just importing the CGI module (or the database module) can take longer > than that... The original performance specification was "...receive up to 2000 calls per minute". I don't believe that means that a call has to be serviced in under 30ms (wall-clock time) but total CPU time would have to be <30ms in order to not fall behind under a constant 2000 requests/second load. So we can probably remove database connection and communication time (i.e. IO-bound components). Still, it's a lot tighter than I though it would be: % time python -c "import SimpleXMLRPCServer; import MySQLdb" real 0m0.144s user 0m0.046s sys 0m0.064s So it's already almost 4x too slow. But I'm running this on Ubuntu, running on VMWare on my 1.6GHz Pentium-M laptop. I would assume that a beefy server would do a lot better. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Importing modules through directory shortcuts on Windows
Recently, I became responsible for maintaining some Python code, which was organized as follows: user/pylib ui ... project2/pylib ui ... project3/pylib ui ... python-packages/user => /user/pylib project2 => /project2/pylib project3 => /project3/pylib The idea is that "python-packages" is added to sys.path and then every project can import the library package from every other project. NOTE: I think that this structure is crazy but I'm just the poor maintenance programmer. Anyway, the problem is that Windows does not have a symlink facility to accommodate this (AFAIK) and the Python import mechanism does not resolve shortcuts. Therefore, I wrote a very simple import hook to get around that problem. If you are interested, the code is here: http://www.sweetapp.com/shortcut/shortcutimporter.py BTW, does anyone else think that this functionality should be part of core Python? Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
ANN: Vancouver Python Workshop
Vancouver Python Workshop = Building on the huge success of the 2004 Vancouver Python Workshop, the Vancouver Python and Zope User Group is pleased to announce the 2006 Vancouver Python Workshop. The conference will begin with keynote addresses on August 4st. Further talks (and tutorials for beginners) will take place on August 5th and 6th. The Vancouver Python Workshop is a community organized conference designed for both the beginner and for the experienced Python programmer with: * tutorials for beginning programmers * advanced lectures for Python experts * case studies of Python in action * after-hours social events * informative keynote speakers * tracks on multimedia, Web development, education and more More information see: http://www.vanpyz.org/conference/ or contact Brian Quinlan at: [EMAIL PROTECTED] Vancouver = In addition to the opportunity to learn and socialize with fellow Pythonistas, the Vancouver Python Workshop also gives visitors the opportunity to visit one of the most extraordinary cities in the world (1). For more information about traveling to Vancouver, see: http://www.vanpyz.org/conference/vancouver.html http://www.tourismvancouver.com http://en.wikipedia.org/wiki/Vancouver Important dates === Talk proposals accepted: May 15th to June 15th Early registration (discounted): May 22nd to June 30th Normal registration: from July 1st Keynotes: August 4th Conference and tutorial dates: August 5th and 6th (1) http://news.bbc.co.uk/2/hi/business/2299119.stm Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Tabs versus Spaces in Source Code
Edward Elliott wrote: > Tab is not 4 spaces. Tab is 1 level of indentation. The confusion that > tabs equals some fixed width, or can/should be set to some fixed width, is > the entire problem hampering their use. It implies that conversion between > tabs and spaces is straightforward when it is not. They are not comparable > entities. The problem with tabs is that people use tabs for alignment e.g. def foo(): ->query = """SELECT * -> -> -> FROM sometable -> -> -> WHERE condition""" Now I change my editor to use 8-space tabs and the code is all messed up. Of course, a very disciplined group of people could be trained to never use tabs except to align with the current block level but, in practice, that doesn't work. Therefore tabs are bad. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
ANN: Vancouver Python Workshop - keynote speakers announced
What's New? === We are pleased to announce the keynote speakers for this year's Vancouver Python Workshop: Guido van Rossum and Jim Hugunin. Guido van Rossum (Google) is the inventor of Python and has managed its growth and development for more than a decade. Guido was awarded the Free Software Foundation Award in 2002 and Dr.Dobb's 1999 Excellence in Programming Award. Today Guido works at Google, spending half of his time on Python. Jim Hugunin (Microsoft) is the creator of IronPython, Jython and Numeric Python. IronPython is Python for the .NET platform and integrates Python into Microsoft's .NET strategy. Jython is Python for the Java platform and was the second production quality implementation of Python. Numeric Python adapts Python to the needs of number crunching applications. Today, Jim works at Microsoft where he helps them adapt the .NET runtime to meet the needs of dynamic languages like Python. About the Vancouver Python Workshop === The conference will begin with keynote addresses on August 4st. Further talks (and tutorials for beginners) will take place on August 5th and 6th. The Vancouver Python Workshop is a community organized conference designed for both the beginner and for the experienced Python programmer with: * tutorials for beginning programmers * advanced lectures for Python experts * case studies of Python in action * after-hours social events * informative keynote speakers * tracks on multimedia, Web development, education and more More information see: http://www.vanpyz.org/conference/ or contact Brian Quinlan at: [EMAIL PROTECTED] Vancouver = In addition to the opportunity to learn and socialize with fellow Pythonistas, the Vancouver Python Workshop also gives visitors the opportunity to visit one of the most extraordinary cities in the world (1). For more information about traveling to Vancouver, see: http://www.vanpyz.org/conference/vancouver.html http://www.tourismvancouver.com http://en.wikipedia.org/wiki/Vancouver Important dates === Talk proposals accepted: May 15th to June 15th Early registration (discounted): May 22nd to June 30th Normal registration: from July 1st Keynotes: August 4th Conference and tutorial dates: August 5th and 6th (1) http://news.bbc.co.uk/2/hi/business/2299119.stm Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Vancouver Python Workshop - registration open
What's New? === Early-bird registration for the Vancouver Python conference is now open. Participants who register before June 30th will receive a substantial discount. To register, see: http://www.vanpyz.org/conference/registration For general conference information, see: http://www.vanpyz.org/conference About the Vancouver Python Workshop === The conference will begin with keynote addresses on August 4st by Guido van Rossum [1] and Jim Hugunin [2]. Further talks (and tutorials for beginners) will take place on August 5th and 6th. The Vancouver Python Workshop is a community organized and designed for both the beginner and for the experienced Python programmer with: * tutorials for beginning programmers * advanced lectures for Python experts * case studies of Python in action * after-hours social events * informative keynote speakers * tracks on multimedia, Web development, education and more More information see: http://www.vanpyz.org/conference/ or contact Brian Quinlan at: [EMAIL PROTECTED] Vancouver = In addition to the opportunity to learn and socialize with fellow Pythonistas, the Vancouver Python Workshop also gives visitors the opportunity to visit one of the most extraordinary cities in the world [3]. For more information about traveling to Vancouver, see: http://www.vanpyz.org/conference/vancouver.html http://www.tourismvancouver.com http://en.wikipedia.org/wiki/Vancouver Important dates === Talk proposals accepted: May 15th to June 15th Early registration (discounted): May 22nd to June 30th Normal registration: from July 1st Keynotes: August 4th Conference and tutorial dates: August 5th and 6th [1] Guido van Rossum (Google) is the inventor of Python and has managed its growth and development for more than a decade. Guido was awarded the Free Software Foundation Award in 2002 and Dr.Dobb's 1999 Excellence in Programming Award. Guido works at Google and spends half of his time on Python. [2] Jim Hugunin (Microsoft) is the creator of numerous innovations that take Python into new application domains. Jim's most recent project, IronPython integrates Python into Microsoft's .NET runtime. Jim's previous project, Jython is Python for the Java runtime and was the second production-quality implementation of Python. Before that, Jim's Numeric Python adapted Python to the needs of number crunching applications. Jim works at Microsoft adapting the .NET runtime to the needs of dynamic languages like Python. [3] http://news.bbc.co.uk/2/hi/business/2299119.stm Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: python3 - the hardest hello world ever ?
Hey Helmut,
Did you try just:
print("Hallo, Süßes Python")
Cheers,
Brian
Helmut Jarausch wrote:
Hi,
do I miss something (I do hope so) or is switching to Python3
really hard for Latin1-users?
My simplest hello world script - which uses a few German
umlaut characters - doesn't look very intuitive.
I have to set an internal property (with leading underscore)
for each output file I'm using - right?
#!/usr/local/bin/python3.0
# _*_ coding: latin1 _*_
import sys
# the following call doesn't do the job
# sys.setfilesystemencoding('latin1')
# but this ugly one (to be done for each output file)
sys.stdout._encoding='latin1'
print("Hallo, Süßes Python")
Thanks for any enlightening on that subject,
Helmut.
--
http://mail.python.org/mailman/listinfo/python-list
Re: XMLRPC - persistent object state on server
Demidov Andrey wrote: class MyClass: def __init__(self, a): self.a = a # and some heavy works which I would like to do once def say(self): return a Change: def say(self): return a to: def say(self): return self.a Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Java-style futures in Python - only better
Hey all,
I've been working on an Java-style futures implementation in Python.
Futures are a way of representing asynchronous operations e.g.
operations that are run in another thread or process. The are are a easy
but powerful way of parallelizing sequential operations. The also
provide a consistent interface across implementations e.g. they can
provide the same interface to threading and to multiprocessing.
For example:
def is_prime(n):
"Return True iff n is prime"
...
def check_primes(numbers):
return map(is_prime, numbers)
Could be parallelized as:
def check_primes(numbers):
# ProcessPoolExecutor will create one worker process
# per CPU if called without arguments. Using threads
# is valueless because of the GIL.
with futures.ProcessPoolExecutor() as executor:
return executor.map(is_prime, numbers)
A more complex example:
def load_url(url, timeout):
return urllib.request.urlopen(url, timeout=timeout).read()
### Download the content of some URLs - ignore failures.
def download_urls(urls, timeout=60):
url_to_content = {}
for url in urls:
try:
url_to_content[url] = load_url(url, timeout=timeout)
except:
pass
return url_to_content
Could be parallelized as:
# Making this a global variable used any many functions ensures that
# the global thread count is kept under control.
executor = futures.ThreadPoolExecutor(50)
def download_urls(urls, timeout=60):
url_to_content = {}
# Run load_url for every url and get the result as a FuturesList.
fs = executor.run_to_futures(
(functools.partial(load_url, url, timeout) for url in urls),
timeout=timeout)
for url, future in zip(urls, fs.successful_futures()):
url_to_content[url] = future.result()
return url_to_content
The Python 3.0+ code is here:
http://code.google.com/p/pythonfutures/source
Any feedback on the API would be very much appreciated!
Cheers,
Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: Java-style futures in Python - only better
Colin J. Williams wrote:
Brian,
Since the word "future" is part of the Python lingo:
A future statement is a directive to the compiler that a particular
module should be compiled using syntax or semantics that will be
available in a specified future release of Python. The future statement
is intended to ease migration to future versions of Python that
introduce incompatible changes to the language. It allows use of the new
features on a per-module basis before the release in which the feature
becomes standard.
Have you given thought to the use of another word?
I named the module "futures" (plural) to try to reduce the potential
confusion with the "__futures__" module.
The concept of a future is fairly well known in CS
[http://en.wikipedia.org/wiki/Future_(programming)] so giving it a
completely different name would be a bit annoying.
Cheers,
Brian
Colin W.
Brian Quinlan wrote:
Hey all,
I've been working on an Java-style futures implementation in Python.
Futures are a way of representing asynchronous operations e.g.
operations that are run in another thread or process. The are are a
easy but powerful way of parallelizing sequential operations. The also
provide a consistent interface across implementations e.g. they can
provide the same interface to threading and to multiprocessing.
For example:
def is_prime(n):
"Return True iff n is prime"
...
def check_primes(numbers):
return map(is_prime, numbers)
Could be parallelized as:
def check_primes(numbers):
# ProcessPoolExecutor will create one worker process
# per CPU if called without arguments. Using threads
# is valueless because of the GIL.
with futures.ProcessPoolExecutor() as executor:
return executor.map(is_prime, numbers)
A more complex example:
def load_url(url, timeout):
return urllib.request.urlopen(url, timeout=timeout).read()
### Download the content of some URLs - ignore failures.
def download_urls(urls, timeout=60):
url_to_content = {}
for url in urls:
try:
url_to_content[url] = load_url(url, timeout=timeout)
except:
pass
return url_to_content
Could be parallelized as:
# Making this a global variable used any many functions ensures that
# the global thread count is kept under control.
executor = futures.ThreadPoolExecutor(50)
def download_urls(urls, timeout=60):
url_to_content = {}
# Run load_url for every url and get the result as a FuturesList.
fs = executor.run_to_futures(
(functools.partial(load_url, url, timeout) for url in urls),
timeout=timeout)
for url, future in zip(urls, fs.successful_futures()):
url_to_content[url] = future.result()
return url_to_content
The Python 3.0+ code is here:
http://code.google.com/p/pythonfutures/source
Any feedback on the API would be very much appreciated!
Cheers,
Brian
--
http://mail.python.org/mailman/listinfo/python-list
Odd closure issue for generators
This is from Python built from the py3k branch: >>> c = (lambda : i for i in range(11, 16)) >>> for q in c: ... print(q()) ... 11 12 13 14 15 >>> # This is expected >>> c = (lambda : i for i in range(11, 16)) >>> d = list(c) >>> for q in d: ... print(q()) ... 15 15 15 15 15 >>> # I was very surprised Looking at the implementation, I see why this happens: >>> c = (lambda : i for i in range(11, 16)) >>> for q in c: ... print(id(q.__closure__[0])) ... 3847792 3847792 3847792 3847792 3847792 >>> # The same closure is used by every lambda But it seems very odd to me and it can lead to some problems that are a real pain in the ass to debug. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Odd closure issue for generators
Gabriel Genellina wrote:
En Thu, 04 Jun 2009 18:40:07 -0300, Brian Quinlan
escribió:
This is from Python built from the py3k branch:
It's not new; same thing happens with 2.x
A closure captures (part of) the enclosing namespace, so names are
resolved in that environment even after the enclosing block has finished
execution.
As always, the name->value evaluation happens when it is required at
runtime, not earlier ("late binding"). So, in the generator expression
(lambda : i for i in range(11, 16)) the 'i' is searched in the enclosing
namespace when the lambda is evaluated, not when the lambda is created.
OK, I talked myself into agreeing that it is better for the generator
comprehension to share a context amongst every enclosed generator
expression (rather than having one context per generator expression) for
consistency reasons.
Thanks!
Cheers,
Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: Odd closure issue for generators
Scott David Daniels wrote: [snipped] When you evaluate a lambda expression, the default args are evaluated, but the expression inside the lambda body is not. When you apply that evaluated lambda expression, the expression inside the lambda body is is evaluated and returned. But that's not really the issue. I knew that the lambda was not evaluated but thought each generator expression got its own context rather than sharing one. Taken in isolation, having one context per expression has more compelling semantics but it is inconsistent with the obvious transliteration of the generator into a loop. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Python and Flaming Thunder
Dave Parker wrote: Or just: If command is "quit" ... Hmmm. In Flaming Thunder, I'm using "is" (and "is an", "is a", etc) for assigning and checking types. For example, to read data from a file and check for errors: Read data from "input.txt". If data is an error then go to ... Hey Dave, Does this mean that Flaming Thunder requires explicit checking rather than offering exceptions? Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Using an DTD not specified in XML file for validation
Hey, I'm trying to figure out how I can validate an XML file using a DTD that isn't specified in the XML file. My code so far is: from xml import sax from xml.sax import sax2exts parser = sax2exts.XMLValParserFactory.make_parser() parser.setContentHandler(handler) parser.setErrorHandler(handler) parser.parse(xml_file) And this works fine if the DTD is specified in the XML file i.e errors are generated for non-compliant entities. But I would like to force the file to be valid according to one other DTD file that is not referenced in the XML file. Anyone know how to do this? Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
New Python logo in high resolution format
Is the new Python logo (i.e. http://python.org/images/python-logo.gif) available someone in a high-resolution format? That would be nice to integrate into websites, T-shirts, etc. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: XMLRPCServer issues
1. Is there on option to get cx_Oracle to return string data as unicode rather than strings objects? XML-RPC aside, dealing with unicode objects might be better than dealing with encoded strings. 2. You might want to transmit integers as strings rather than use the XML-RPC integer type (which is limited to numbers between -2147483648 and 2147483647). Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: New Python logo in high resolution format
The new Python logo is available in high-resolution format here: http://tinyurl.com/n4rge Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: tips for this exercise?
John Salerno wrote: > I'm working on another exercise now about generating random numbers for > the lottery. What I want to do is write a function that picks 5 random > numbers from 1-53 and returns them. Here's what I have so far: > > numbers = range(1, 54) > > def genNumbers(): > for x in range(5): > fiveNumbers = [] > number = random.choice(numbers) > numbers.remove(number) > fiveNumbers = fiveNumbers.append(number) > return fiveNumbers > > Other than being sort of ugly, this also has the side effect of actually > editing the original list, which I don't want since I will want to > generate more than one set of numbers. > > Is there a better way to extract a certain number of items from a list > (so maybe I don't need the for loop)? Is a list even the right type to > use, since it gets edited in place? Perhaps a set? I would just write the function like this: def genNumbers(): shuffle_nums = numbers[:]# copy the list to preserve the orginal # order (if it matters) random.shuffle(shuffle_nums) # shuffle the entire list return shuffle_nums[:5] # return the first 5 elements Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: Working with files in a SimpleXMLRPCServver
Jose Carlos Balderas Alberico wrote: > I'm setting up a server accepting XML-RPC calls using the > SimpleXMLRPCServer class. Basically, what I have to do is send a > zip-compressed file to the server, have the server unzip it and process > it, after processing it the server is supposed to zip the file again, > and send it back to the client. Using an XML-RPC server is overkill if you are just sending a single file and processing the result. You could just use a HTTP server. And Python has a library for doing zip processing (zlib), so you don't need to bother creating a file just to unzip your data. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: download for official Python logo artwork?
has wrote: > Anyone know where I can find source artwork, preferably vector-based, > for python.org's new 'ying-yang' snake icon? I think it's hiding. > Thanks. > I don't know how office it is, but you can get the artwork here: http://tinyurl.com/n4rge Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: XML-RPC server via xinetd
Jos Vos wrote: > The problem is that I do not see how to let an SimpleXMLRPCServer > instance *not* bind to a port or what other class I can use to just > build a XML-RPC request handler reading/writing from stdin/stdout, > i.s.o. carrying all the server class stuff with it. I think that the problem here is that we are confusing transport with request handling. If you take a look at CGIXMLRPCRequestHandler (http://docs.python.org/lib/node564.html), you will see an example of how to write an XMLRPCRequestHandler without HTTP. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: XML-RPC server via xinetd
Jos Vos wrote:
> On Mon, Apr 17, 2006 at 03:30:04AM -0500, Nick Craig-Wood wrote:
>
>> UTSL ;-)
>>
>> Look at /usr/lib/python2.4/SimpleXMLRPCServer.py (adjust as per your
>> distro) and in particular the definition of the CGIXMLRPCRequestHandler
>> class.
>
> I did this before posting my question, in fact, but I did not look
> good enough maybe, as at first sight I thought tghe CGI... class
> would be too CGI-specific (it looks for environment variables etc.
> given by the HTTP server), but maybe it's good enough.
I don't know exactly what your usage pattern is, but you might be able
to use SimpleXMLRPCDispatcher directly e.g.
>>> s = SimpleXMLRPCDispatcher()
>>> s.register_function(pow)
>>> s._marshaled_dispatch('http://mail.python.org/mailman/listinfo/python-list
Re: SimpleXMLRPCServer clobbering sys.stderr? (2.5.2)
Do you have some code that we could see that provokes the problem? Cheers, Brian Joseph Turian wrote: I was having a mysterious problem with SimpleXMLRPCServer. (I am using Python 2.5.2) The request handlers were sometimes failing without any error message to the log output. What I discovered was perplexing. I had some 'print' statements in the handers that, assuming the request would be handled, would print just fine. When I switched to 'print >> sys.stderr', the request handlers would just fail completely, and not make the sys.stderr output that I desired. It seems that SimpleXMLRPCServer is clobbering stderr in some bizarre and silent-error-causing way. I can't really find any documentation of explanation of this phenomenon. Could someone please illuminate it for me? Best, Joseph -- http://mail.python.org/mailman/listinfo/python-list
multiprocessing deadlock
My test reduction:
import multiprocessing
import queue
def _process_worker(q):
while True:
try:
something = q.get(block=True, timeout=0.1)
except queue.Empty:
return
else:
print('Grabbed item from queue:', something)
def _make_some_processes(q):
processes = []
for _ in range(10):
p = multiprocessing.Process(target=_process_worker, args=(q,))
p.start()
processes.append(p)
return processes
def _do(i):
print('Run:', i)
q = multiprocessing.Queue()
for j in range(30):
q.put(i*30+j)
processes = _make_some_processes(q)
while not q.empty():
pass
#The deadlock only occurs on Mac OS X and only when these lines
#are commented out:
#for p in processes:
#p.join()
for i in range(100):
_do(i)
--
Output (on Mac OS X using the svn version of py3k):
% ~/bin/python3.2 moprocessmoproblems.py
Run: 0
Grabbed item from queue: 0
Grabbed item from queue: 1
Grabbed item from queue: 2
...
Grabbed item from queue: 29
Run: 1
At this point the script produces no additional output. If I uncomment
the lines above then the script produces the expected output. I don't
see any docs that would explain this problem and I don't know what the
rule would be e.g. you just join every process that uses a queue before
the queue is garbage collected.
Any ideas why this is happening?
Cheers,
Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: multiprocessing deadlock
On 24 Oct 2009, at 00:02, paulC wrote:
On Oct 23, 3:18 am, Brian Quinlan wrote:
My test reduction:
import multiprocessing
import queue
def _process_worker(q):
while True:
try:
something = q.get(block=True, timeout=0.1)
except queue.Empty:
return
else:
print('Grabbed item from queue:', something)
def _make_some_processes(q):
processes = []
for _ in range(10):
p = multiprocessing.Process(target=_process_worker,
args=(q,))
p.start()
processes.append(p)
return processes
def _do(i):
print('Run:', i)
q = multiprocessing.Queue()
for j in range(30):
q.put(i*30+j)
processes = _make_some_processes(q)
while not q.empty():
pass
#The deadlock only occurs on Mac OS X and only when these lines
#are commented out:
#for p in processes:
#p.join()
for i in range(100):
_do(i)
--
Output (on Mac OS X using the svn version of py3k):
% ~/bin/python3.2 moprocessmoproblems.py
Run: 0
Grabbed item from queue: 0
Grabbed item from queue: 1
Grabbed item from queue: 2
...
Grabbed item from queue: 29
Run: 1
At this point the script produces no additional output. If I
uncomment
the lines above then the script produces the expected output. I don't
see any docs that would explain this problem and I don't know what
the
rule would be e.g. you just join every process that uses a queue
before
the queue is garbage collected.
Any ideas why this is happening?
Cheers,
Brian
I can't promise a definitive answer but looking at the doc.s:-
isAlive()
Return whether the thread is alive.
Roughly, a thread is alive from the moment the start() method
returns until its run() method terminates. The module function
enumerate() returns a list of all alive threads.
I guess that the word 'roughly' indicates that returning from the
start
() call does not mean that all the threads have actually started, and
so calling join is illegal. Try calling isAlive on all the threads
before returning from _make_some_processes.
Regards, Paul C.
--
http://mail.python.org/mailman/listinfo/python-list
Hey Paul,
I guess I was unclear in my explanation - the deadlock only happens
when I *don't* call join.
Cheers,
Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: multiprocessing deadlock
On 24 Oct 2009, at 06:01, paulC wrote: Hey Paul, I guess I was unclear in my explanation - the deadlock only happens when I *don't* call join. Cheers, Brian Whoops, my bad. Have you tried replacing prints with writing a another output Queue? I'm wondering if sys.stdout has a problem. Removing the print from the subprocess doesn't prevent the deadlock. Cheers, Brian -- http://mail.python.org/mailman/listinfo/python-list
Re: multiprocessing deadlock
On 24 Oct 2009, at 14:10, Gabriel Genellina wrote:
En Thu, 22 Oct 2009 23:18:32 -0300, Brian Quinlan
escribió:
I don't like a few things in the code:
def _do(i):
print('Run:', i)
q = multiprocessing.Queue()
for j in range(30):
q.put(i*30+j)
processes = _make_some_processes(q)
while not q.empty():
pass
I'd use time.sleep(0.1) or something instead of this busy wait, but
see below.
This isn't my actual code, it is a simplification of my code designed
to minimally demonstrate a possible bug in multiprocessing.
#The deadlock only occurs on Mac OS X and only when these lines
#are commented out:
#for p in processes:
#p.join()
I don't know how multiprocessing deals with it, but if you don't
join() a process it may become a zombie, so it's probably better to
always join them. In that case I'd just remove the wait for
q.empty() completely.
I'm actually not looking for workarounds. I want to know if this is a
multiprocessing bug or if I am misunderstanding the multiprocessing
docs somehow and my demonstrated usage pattern is somehow incorrect.
Cheers,
Brian
for i in range(100):
_do(i)
Those lines should be guarded with: if __name__ == '__main__':
I don't know if fixing those things will fix your problem, but at
least the code will look neater...
--
Gabriel Genellina
--
http://mail.python.org/mailman/listinfo/python-list
--
http://mail.python.org/mailman/listinfo/python-list
Re: multiprocessing deadlock
On 24 Oct 2009, at 19:49, Gabriel Genellina wrote: En Sat, 24 Oct 2009 02:48:38 -0300, Brian Quinlan escribió: On 24 Oct 2009, at 14:10, Gabriel Genellina wrote: En Thu, 22 Oct 2009 23:18:32 -0300, Brian Quinlan > escribió: I don't like a few things in the code: I'm actually not looking for workarounds. I want to know if this is a multiprocessing bug or if I am misunderstanding the multiprocessing docs somehow and my demonstrated usage pattern is somehow incorrect. Those aren't really workarounds, but things to consider when trying to narrow down what's causing the problem. The example is rather long as it is, and it's hard to tell what's wrong since there are many places thay might fail. I agree that the multiprocessing implementation is complex is there are a lot of spinning wheels. At this point, since no one has pointed out how I am misusing the module, I think that I'll just file a bug. The busy wait might be relevant, or not; having a thousand zombie processes might be relevant, or not. According to the docs: """On Unix when a process finishes but has not been joined it becomes a zombie. There should never be very many because each time a new process starts (or active_children() is called) all completed processes which have not yet been joined will be joined. Also calling a finished process’s Process.is_alive() will join the process. Even so it is probably good practice to explicitly join all the processes that you start.""" Cheers, Brian-- http://mail.python.org/mailman/listinfo/python-list
Re: multiprocessing deadlock
On 24 Oct 2009, at 21:37, larudwer wrote:
"Brian Quinlan" schrieb im Newsbeitrag
news:mailman.1895.1256264717.2807.python-l...@python.org...
Any ideas why this is happening?
Cheers,
Brian
IMHO your code is buggy. You run in an typical race condition.
consider following part in your code:
def _make_some_processes(q):
processes = []
for _ in range(10):
p = multiprocessing.Process(target=_process_worker, args=(q,))
p.start()
processes.append(p)
return processes
p.start() may start an process right now, in 5 seconds or an week
later,
depending on how the scheduler of your OS works.
Agreed.
Since all your processes are working on the same queue it is -- very
--
likely that the first process got started, processed all the input and
finished, while all the others haven't even got started.
Agreed.
Though your first
process exits, and your main process also exits, because the queue
is empty
now ;).
The main process shouldn't (and doesn't exit) - the _do function exits
(with some processes possibly still running) and the next iteration in
for i in range(100):
_do(i)
is evaluated.
while not q.empty():
pass
If you where using p.join() your main process wourd terminate when
the last
process terminates !
That's an different exit condition!
When you say "your main process would terminate", you mean that the
_do function would exit, right? Because process.join() has nothing to
do with terminating the calling process - it just blocks until process
terminates.
When the main process terminates all the garbage collection fun
happens. I
hope you don't wonder that your Queue and the underlaying pipe got
closed
and collected!
I expected the queue and underlying queue and pipe to get collected.
Well now that all the work has been done, your OS may remember that
someone
sometimes in the past told him to start an process.
Sure, that could happen at this stage. Are you saying that it is the
user of the multiprocessing module's responsibility to ensure that the
queue is not collected in the parent process until all the child
processes using it have exited? Actually, causing the queues to never
be collected fixes the deadlock:
+ p = []
def _do(i):
print('Run:', i)
q = multiprocessing.Queue()
+ p.append(q)
print('Created queue')
for j in range(30):
q.put(i*30+j)
processes = _make_some_processes(q)
print('Created processes')
while not q.empty():
pass
print('Q is empty')
This behavior is counter-intuitive and, as far as I can tell, not
documented anywhere. So it feels like a bug.
Cheers,
Brian
def _process_worker(q):
while True:
try:
something = q.get(block=True, timeout=0.1)
except queue.Empty:
return
else:
print('Grabbed item from queue:', something)
The line
something = q.get(block=True, timeout=0.1)
should cause some kind of runtime error because q is already
collected at
that time.
Depending on your luck and the OS this bug may be handled or not.
Obviously
you are not lucky on OSX ;)
That's what i think happens.
--
http://mail.python.org/mailman/listinfo/python-list
--
http://mail.python.org/mailman/listinfo/python-list
futures - a new package for asynchronous execution
Hey all,
I recently implemented a package that I'd like to have include in the
Python 3.x standard library (and maybe Python 2.x) and I'd love to
have the feedback of this list.
The basic idea is to implement an asynchronous execution method
patterned heavily on java.util.concurrent (but less lame because
Python has functions as first-class objects). Here is a fairly
advanced example:
import futures
import functools
import urllib.request
URLS = [
'http://www.foxnews.com/',
'http://www.cnn.com/',
'http://europe.wsj.com/',
'http://www.bbc.co.uk/',
'http://some-made-up-domain.com/']
def load_url(url, timeout):
return urllib.request.urlopen(url, timeout=timeout).read()
# Use a thread pool with 5 threads to download the URLs. Using a pool
# of processes would involve changing the initialization to:
# with futures.ProcessPoolExecutor(max_processes=5) as executor
with futures.ThreadPoolExecutor(max_threads=5) as executor:
future_list = executor.run_to_futures(
[functools.partial(load_url, url, 30) for url in URLS])
# Check the results of each future.
for url, future in zip(URLS, future_list):
if future.exception() is not None:
print('%r generated an exception: %s' % (url,
future.exception()))
else:
print('%r page is %d bytes' % (url, len(future.result(
In this example, executor.run_to_futures() returns only when every url
has been retrieved but it is possible to return immediately, on the
first completion or on the first failure depending on the desired work
pattern.
The complete docs are here:
http://sweetapp.com/futures/
A draft PEP is here:
http://code.google.com/p/pythonfutures/source/browse/trunk/PEP.txt
And the code is here:
http://pypi.python.org/pypi/futures3/
All feedback appreciated!
Cheers,
Brian
--
http://mail.python.org/mailman/listinfo/python-list
Re: why does "help(import)" not work?
Hi Robert,
help() is just a regular function that must be called with correct
Python syntax and the import keyword is not allowed in an argument list.
The correct syntax is:
help('import')
Cheers,
Brian
On 6 Nov 2009, at 20:56, Robert P. J. Day wrote:
i'm sure there's a painfully obvious answer to this, but is there a
reason i can't do:
help(import)
File "", line 1
help(import)
^
SyntaxError: invalid syntax
on the other hand, i can certainly go into "help()" and type
"import" to get that help. it seems counter-intuitive to have the
first variation fail but the second succeed.
what is the rule for this in python3?
rday
--
=
=
==
Robert P. J. Day Waterloo, Ontario,
CANADA
Linux Consulting, Training and Kernel Pedantry.
Web page: http://crashcourse.ca
Twitter: http://twitter.com/rpjday
=
=
==
--
http://mail.python.org/mailman/listinfo/python-list
--
http://mail.python.org/mailman/listinfo/python-list
