Re: attributes, properties, and accessors -- philosophy

[Bruno Desthuilliers]

Ethan Furman a écrit :
The problem I have with properties is my typing.  I'll end up assigning 
to an attribute, but get the spelling slightly wrong (capitalized, or 
missing an underscore -- non-obvious things when bug-hunting), so now I 
have an extra attribute which of course has zero effect on what I'm 
trying to do and I start getting wierd results like viewing deleted 
records when I *know* I set useDeleted = False... 30 minutes later I 
notice it was /supposed/ to be use_deleted.  *sigh*


So -- to keep myself out of trouble -- I have started coding such things 
as, for example:


result = table.use_deleted()   # returns True or False
table.use_deleted(False)   # skip deleted records

instead of

result = table.use_deleted
table.use_deleted = False

My question:  is this [ severely | mildly | not at all ] un-pythonic?


Definitly and totally unpythonic. The first solution to your problem is 
to stick to standard naming conventions. If this is not enough, Chris 
pointed you to really useful tools. Also, you can override __setattr__ 
to catch such errors - at least during the coding/debug phase.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Where to put the error handing test?

[Bruno Desthuilliers]

alex23 a écrit :

On Nov 24, 1:15 pm, Peng Yu  wrote:

Suppose that I have function f() that calls g(), I can put a test on
the argument 'x' in either g() or f(). I'm wondering what is the
common practice.

If I put the test in f(), then g() becomes more efficient when other
code call g() and guarantee x will pass the test even though the test
code in not in g(). But there might be some caller of g() that pass an
'x' that might not pass the test, if there were the test in g().


What you should try to do is make each function as self-contained as
possible. f() shouldn't have to know what is a valid argument for g(),
that's the responsibility of g(). 


There's no such clear-cut IMHO - it really depends on the context. If f 
is a user-interface function - a function that deals with user inputs in 
whatever form - and g is a domain-specific library function, then it's 
f's responsability to validate user inputs before calling on g (_and_ of 
course to deal with any exception raised withing g).


As a general rule, "defensive code" should go at the interface level - 
program's inputs of course, but also, sometimes, at "sub-systems" 
boundaries.

--
http://mail.python.org/mailman/listinfo/python-list

Re: lxml 2.2.4 for Python 2.6

[Sérgio Monteiro Basto]

Hi,

Srijit Kumar Bhadra wrote:

> Is there any reason why lxml-2.2.4-py2.6-win32.egg  (md5) or
> lxml-2.2.4.win32-py2.6.exe is not available?
> 
> Best regards,
> /Srijit

maybe ask on lxml Mailing List , should be more appropriated 

Sérgio M. B.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Where to put the error handing test?

[Paul Miller]

On Mon, 23 Nov 2009 22:27:24 -0800, alex23 wrote:

> As a very rough example:
> 
> def g(x):
> try:
> assert isinstance(x, int)
> except AssertionError:
> raise TypeError, "excepted int, got %s" % type(x)
> # ... function code goes here
> 
> def f(x):
> try:
> g(x)
> except TypeError:
> # handle the problem here
> # ... function code goes here

I know you say this is a "very rough" example, but, generally you don't 
want to do this kind of "type checking" with isinstance.  Rather, it's 
better to just simply manipulate x as if it were an integer and rely on 
Python to check to see if x supports the operations you're trying to do 
with it.  For instance, say we have

def g(x):
return x * x

def f(x):
return g(x) + 2

If you try to pass any value to either of these functions that doesn't 
support the required operations, Python itself will complain with a 
TypeError.  Since the interpreter needs to do this check *anyway*, 
there's no real sense in repeating it manually by checking isinstance.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: xmlrpc idea for getting around the GIL

[sturlamolden]

On 22 Nov, 22:38, Patrick Stinson 
wrote:

> Has anyone every tried wrapping the CPython lib into a daemon with an
> RPC mechanism in order to move the GIL out of the process?

> I imagine this is how the multiprocessing module works.

It does not.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Where to put the error handing test?

[Dave Angel]

Peng Yu wrote:

On Mon, Nov 23, 2009 at 9:44 PM, Lie Ryan  wrote:
  

Peng Yu wrote:


Suppose that I have function f() that calls g(), I can put a test on
the argument 'x' in either g() or f(). I'm wondering what is the
common practice.

My thought is that if I put the test in g(x), the code of g(x) is
safer, but the test is not necessary when g() is called by h().

If I put the test in f(), then g() becomes more efficient when other
code call g() and guarantee x will pass the test even though the test
code in not in g(). But there might be some caller of g() that pass an
'x' that might not pass the test, if there were the test in g().
  

Typically, you test for x as early as possible, e.g. just after user input
(or file or url load or whatever). After that test, you can (or should be
able to) assume that all function calls will always be called with the
correct argument. This is the ideal situation, it's not always easy to do.

In any case though, don't optimize early.



Let's suppose that g() is refactored out from f() and is call by not
only f() but other functions, and g() is likely to be called by new
functions.

If I don't optimize early, I should put the test in g(), rather than f(), right?

  
Your question is so open-ended as to be unanswerable.  All we should do 
in this case is supply some guidelines so you can guess which one might 
apply in your particular case.


You could be referring to a test that triggers alternate handling.  Or 
you could be referring to a test that notices bad input by a user, or 
bad data from an untrusted source.  Or you could be referring to a test 
that discovers bugs in your code.  And there are variations of these, 
depending on whether your user is also writing code (eval, or even 
import of user-supplied mixins), etc.


The first thing that's needed in the function g() is a docstring, 
defining what inputs it expects, and what it'll do with them.  Then if 
it gets any input that doesn't meet those requirements, it might throw 
an exception.  Or it might just get an arbitrary result.  That's all up 
to the docstring.  Without any documentation, nothing is correct.


Functions that are only called by trusted code need not have explicit 
tests on their inputs, since you're writing it all.  Part of debugging 
is catching those cases where f () can pass bad data to g().  If it's 
caused because bad data is passed to f(), then you have a bug in that 
caller.  Eventually, you get to the user.  If the bad data comes from 
the user, it should be caught as soon as possible, and feedback supplied 
right then.


assert() ought to be the correct way to add tests in g() that test 
whether there's such a bug in f().  Unfortunately, in CPython it 
defaults to debug mode, so scripts that are run will execute those tests 
by default.  Consequently, people leave them out, to avoid slowing down 
code.




It comes down to trust.  If you throw the code together without a test 
suite, you'll be a long time finding all the bugs in non-trivial code.  
So add lots of defensive tests throughout the code, and pretend that's 
equivalent to a good test system.  If you're writing a library to be 
used by others, then define your public interfaces with exceptions for 
any invalid code, and write careful documentation describing what's 
invalid.  And if you're writing an end-user application, test their 
input as soon as you get it, so none of the rest of the application ever 
gets "invalid" data.



DaveA
--
http://mail.python.org/mailman/listinfo/python-list

Re: xmlrpc idea for getting around the GIL

[sturlamolden]

On 23 Nov, 22:02, Patrick Stinson 
wrote:

> What I meant was that I am *not allowed* to make calls to the CPython
> API from the threads I currently have because these threads are high
> priority and are never allowed to make blocking calls. Fortunately,
> each of these threads can have a completely separate interpreter,

This seems confused. What would they do with the interpreter if they
cannot call the CPython API?


> My question was whether or not anyone has done anything like this in
> C/C++.

I have programmed parallel numerical software in Python, including on-
line signal processing. I have yet to find the GIL gets in my way.

Usually people complaining about the GIL does not know what they are
talking about.



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: xmlrpc idea for getting around the GIL

[Daniel Fetchinson]

>> icating) the multiprocessing module would be ideal.
>>> > The problem is that the OP has a embedded application running threads.
>>> > multiprocssing doesn't help there.
>>>
>>> that's right. I cannot make CPython calls from my original C-based
>>> threads.
>>
>>
>> It's quite possible to do that.  A thread started from C can make
>> calls to Python if it first calls PyGILState_Ensure, although you'd
>> have to make sure that the Python interpreter has been previously
>> initialized.  See PEP 311 for details.
>>
>> http://www.python.org/dev/peps/pep-0311/
>>
>> I also suggest that if you want people to be more receptive to write
>> your replies after the quoted text.  That is the custom in this
>> newsgroup/mailing list.
>>
>
> What I meant was that I am *not allowed* to make calls to the CPython
> API from the threads I currently have because these threads are high
> priority and are never allowed to make blocking calls. Fortunately,
> each of these threads can have a completely separate interpreter, so
> my idea was to create a daemon process for each thread. This daemon
> would contain the libpython symbols and would make the CPython calls.
> This would keep my current threads from having to contend over the
> single GIL.
>
> My question was whether or not anyone has done anything like this in
> C/C++. This situation is slightly unique in that I am trying to
> maintain separate interpreters within a single app (which is currently
> kind of hacked out using a single interpreter, but it's ugly), but I
> could see how this sort of thing would be useful for other C/C++ apps
> that implement an embedded scripting engine.
>
> My reference to multiprocessing was based on the idea that the library
> hides the details fo the process management, shared memory, and rpc
> mechanisms. Again, I can't use multiprocessing because it runs *in*
> python I need this to be implemented *outside* of python to avoid
> acquiring the GIL. complex, I know.
>
> Naturally, the most intimidating part of perusing this kind of idea is
> the overhead of the processess management and the RPC. Further,
> libpython executes callable objects using C function pointers, and I
> can't think of a way that this would be able to re-gain access to the
> original app's respective functions if the interpreter was living in
> another processes.
>
> That's not to mention the impossibility of debugging.
>
> Damn you, Gil.


By the way, you might be interested in this thread as well:

http://groups.google.com/group/comp.lang.python/browse_thread/thread/2d537ad8df9dab67/cc4cb2b493c98170

HTH,
Daniel


-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: xmlrpc idea for getting around the GIL

[sturlamolden]

On 24 Nov, 12:15, Daniel Fetchinson  wrote:

> By the way, you might be interested in this thread as well:
>
> http://groups.google.com/group/comp.lang.python/browse_thread/thread/...

It is Windows specific, and it does not work with extension modules.

Yes we can embed multiple interpreters just by making multiple copies
of Python26.dll. It's an ugly hack and not one I'd recommend for
production servers.


-- 
http://mail.python.org/mailman/listinfo/python-list

pointless musings on performance

[mk]

#!/usr/local/bin/python

import timeit


def pythonic():
nonevar = None
zerovar = 0
for x in range(100):
if nonevar:
pass
if zerovar:
pass

def unpythonic():
nonevar = None
zerovar = 0
for x in range(100):
if nonevar is not None:
pass
if zerovar > 0:
pass

for f in [pythonic, unpythonic]:
print f.func_name, timeit.timeit(f, number=10)



# ./t.py
pythonic 2.13092803955
unpythonic 2.82064604759

Decidedly counterintuitive: are there special optimizations for "if 
nonevar:" type of statements in cpython implementation?



regards,
mk


--
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

[MRAB]

mk wrote:

#!/usr/local/bin/python

import timeit


def pythonic():
nonevar = None
zerovar = 0
for x in range(100):
if nonevar:
pass
if zerovar:
pass

def unpythonic():
nonevar = None
zerovar = 0
for x in range(100):
if nonevar is not None:
pass
if zerovar > 0:
pass

for f in [pythonic, unpythonic]:
print f.func_name, timeit.timeit(f, number=10)



# ./t.py
pythonic 2.13092803955
unpythonic 2.82064604759

Decidedly counterintuitive: are there special optimizations for "if 
nonevar:" type of statements in cpython implementation?



In what way is it counterintuitive? In 'pythonic' the conditions are
simpler, less work is being done, therefore it's faster.
--
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

[Rob Williscroft]

mk wrote in news:mailman.915.1259064240.2873.python-l...@python.org in 
comp.lang.python:

> 
> def pythonic():
 
> def unpythonic():
 
 
> Decidedly counterintuitive: are there special optimizations for "if 
> nonevar:" type of statements in cpython implementation?
> 

from dis import dis

dis( unpythonic )

18  31 LOAD_FAST0 (nonevar)
 34 LOAD_CONST   0 (None)
 37 COMPARE_OP   9 (is not)
 40 JUMP_IF_FALSE4 (to 47)

dis( pythonic )

11  31 LOAD_FAST0 (nonevar)
 34 JUMP_IF_FALSE4 (to 41)
 
Rob.
 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Where to put the error handing test?

[Steven D'Aprano]

On Mon, 23 Nov 2009 21:15:48 -0600, Peng Yu wrote:

> Suppose that I have function f() that calls g(), I can put a test on the
> argument 'x' in either g() or f(). I'm wondering what is the common
> practice.
> 
> My thought is that if I put the test in g(x), the code of g(x) is safer,
> but the test is not necessary when g() is called by h().

If the function g requires the test, then put it in g. If it does not 
require the test, then don't put it in g.

If the test is only required by f, then it belongs in f.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

[Chris Rebert]

On Tue, Nov 24, 2009 at 4:31 AM, Rob Williscroft  wrote:
> mk wrote in news:mailman.915.1259064240.2873.python-l...@python.org in
> comp.lang.python:
>
>>
>> def pythonic():
>
>> def unpythonic():
>
>
>> Decidedly counterintuitive: are there special optimizations for "if
>> nonevar:" type of statements in cpython implementation?
>>
>
> from dis import dis
>
> dis( unpythonic )
>
> 18          31 LOAD_FAST                0 (nonevar)
>             34 LOAD_CONST               0 (None)
>             37 COMPARE_OP               9 (is not)
>             40 JUMP_IF_FALSE            4 (to 47)
>
> dis( pythonic )
>
> 11          31 LOAD_FAST                0 (nonevar)
>             34 JUMP_IF_FALSE            4 (to 41)

In other words, CPython doesn't happen to optimize `if nonevar is not
None` as much as it theoretically could (which would essentially
require a JUMP_IF_NONE opcode). Since CPython isn't known for doing
fancy optimizations, this isn't surprising.

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Any elegant way to construct the complete $k$-partite graph in Python?

[Malte Helmert]

Paul Miller wrote:
> On Mon, 23 Nov 2009 19:57:05 -0800, Richard Thomas wrote:
> 
>> Not sure exactly how you're representing graphs, this seems like the
>> simplest way of listing the edges.
>>
>> def complete_partite(*sizes):
>> total = sum(sizes)
>> nodes, edges = range(total), []
>> for group in xrange(len(sizes)):
>> low = sum(sizes[:group-1])
>> high = sum(sizes[:group])

I think this has a conceptual off-by-one error. Add

   print group, low, high

to see what I mean (especially the first iteration). It still works, but
I think this would be clearer:

   low = sum(sizes[:group])
   high = sum(sizes[:group + 1])

or to avoid doing essentially the same summation twice:

   low = sum(sizes[:group])
   high = low + sizes[group]

>> edges.extend((i, j) for i in xrange(low, high)
>> for j in xrange(high, total))
>> return nodes, edges

Here's a variant that uses a running total instead of recomputing the
sum in every iteration, thus getting rid of xrange(len(...)).

def complete_partite(*sizes):
total = sum(sizes)
nodes, edges = range(total), []
curr_total = 0
for size in sizes:
edges.extend((i, j) for i in xrange(curr_total, curr_total+size)
for j in xrange(curr_total+size, total))
curr_total += size
return nodes, edges

Finally, here is a variant that is a bit shorter because it produces the
edges in a different way and hence gets rid of the need for knowing the
total up front and uses total as running total instead. It has the
drawback of not generating the edges in ascending order though, so I
think the previous one is nicer:

def complete_partite(*sizes):
total, edges = 0, []
for size in sizes:
edges.extend((i, j) for i in xrange(total)
for j in xrange(total, total + size))
total += size
return range(total), edges

Finally, here's a variation on the same theme:

def complete_partite(*sizes):
nodes, edges = [], []
for size in sizes:
partition = xrange(len(nodes), len(nodes) + size)
edges.extend((i, j) for i in nodes for j in partition)
nodes.extend(partition)
return nodes, edges

Malte

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: UnicodeDecodeError? Argh! Nothing works! I'm tired and hurting and...

[Steven D'Aprano]

On Mon, 23 Nov 2009 22:06:29 +0100, Alf P. Steinbach wrote:


> 6. Googling, yes, it seems Thunderbird has a habit of "forgetting"
> mails. But they're really there after all. It's just the index that's
> screwed up.
[...]
> And I'm hesitant to just delete index file, hoping that it'll rebuild.

Right-click on the mailbox and choose "Rebuild Index".

If you're particularly paranoid, and you probably should be, make a 
backup copy of the entire mail folder first.

http://kb.mozillazine.org/Compacting_folders
http://kb.mozillazine.org/Recover_messages_from_a_corrupt_folder
http://kb.mozillazine.org/Disappearing_mail


Good grief, it's about six weeks away from 2010 and Thunderbird still 
uses mbox as it's default mail box format. Hello, the nineties called, 
they want their mail formats back! Are the tbird developers on crack or 
something? I can't believe that they're still using that crappy format.

No, I tell a lie. I can believe it far too well.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Beginning Question about Python functions, parameters...

[Peter Otten]

Terry Reedy wrote:

> astral orange wrote:
> 
>> As far as the program. I did add print statements such as print
>> (MyNames) and got back:
>> 
>> {'middle': {}, 'last': {'Smith': ['John Larry Smith']}, 'first': {}}
> 
> Hmmm, as I understood the code, either that should be ... 'last': {} ...
> before the first store(), as you seem to be thinking below, or after the
> first store(),
> 
> {'middle': {'Larry': ['John Larry Smith'],
>   'last':   {'Smith': ['John Larry Smith'],
>   'first':  {'John' " ['John Larry Smith']}
> 
> or the same with [['John','Larry','Smith']] for each entry (do not
> remember exactly what was stored. Maybe a typo.

That's a bug in the store() function

# as posted
def store(data, full_name):
names = full_name.split()
if len(names) == 2: names.insert(1, '')
labels = 'first', 'middle', 'last'
for label, name in zip(labels, names):
people = lookup(data, label, name)
if people:
people.append(full_name)
else:
data[label][name] = [full_name]

# what was probably intended
def store(data, full_name):
names = full_name.split()
if len(names) == 2: names.insert(1, '')
labels = 'first', 'middle', 'last'
for label, name in zip(labels, names):
people = lookup(data, label, name)
if people:
people.append(full_name)
else:
data[label][name] = [full_name]


Peter

-- 
http://mail.python.org/mailman/listinfo/python-list

Recall: How to log messages _only once_ from all modules ?

[Barak, Ron]

Barak, Ron would like to recall the message, "How to log messages _only once_ 
from all modules ?".
-- 
http://mail.python.org/mailman/listinfo/python-list

How to log messages _only once_ from all modules ?

[Barak, Ron]

Hi,

I'm trying to add the logging module to my application, but I seem to be 
missing something.
My application (a wxPython one) has a main script that calls various helper 
classes.
I want the log messages from all modules to go to one central log file.

When I implement logging, I think that due to preparation, I get the same 
message more than once.

Here's an example:

$ cat -n server.py
 1  import logging
 2  import logging.handlers
 3
 4  class Server():
 5  def __init__(self):
 6  self.client_logger = logging.getLogger("client")
 7  self.client_logger.setLevel(logging.DEBUG)
 8  h = logging.FileHandler("client.log")
 9  h.setLevel(logging.DEBUG)
10  formatter = logging.Formatter("%(asctime)s %(name)-12s 
%(levelname)-8s %(message)s")
11  h.setFormatter(formatter)
12  self.client_logger.addHandler(h)
13
14  def util(self):
15  self.client_logger.warning('This message comes from Server 
module')

$ cat -n client.py
 1  import logging
 2  import logging.handlers
 3  from server import Server
 4
 5  class Client():
 6  def __init__(self):
 7  self.client_logger = logging.getLogger("client")
 8  self.client_logger.setLevel(logging.DEBUG)
 9  h = logging.FileHandler("client.log")
10  h.setLevel(logging.DEBUG)
11  formatter = logging.Formatter("%(asctime)s %(name)-12s 
%(levelname)-8s %(message)s")
12  h.setFormatter(formatter)
13  self.client_logger.addHandler(h)
14
15  def client_test(self):
16  self.client_logger.warning("This message comes from Client 
module")
17
18  if __name__ == "__main__":
19  ser = Server()
20  cli = Client()
21  ser.util()
22  cli.client_test()
$ rm client.log ; python client.py ; cat client.log
2009-11-24 14:40:39,762 client   WARNING  This message comes from Server 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Server 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Client 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Client 
module
Googling and reading http://docs.python.org/library/logging.html didn't 
enlighten me.

Could you suggest what should I change in the above scripts so that the log 
messages would appear only once ?

Thanks,
Ron.

<><>

client.log
Description: client.log


client.py
Description: client.py


server.py
Description: server.py
-- 
http://mail.python.org/mailman/listinfo/python-list

How to log messages _only once_ from all modules ?

[Barak, Ron]

Hi,

I'm trying to add the logging module to my application, but I seem to be 
missing something.
My application (a wxPython one) has a main script that calls various helper 
classes.
I want the log messages from all modules to go to one central log file.

When I implement logging, I think that due to preparation, I get the same 
message more than once.

Here's an example:

$ cat -n server.py
 1  import logging
 2  import logging.handlers
 3
 4  class Server():
 5  def __init__(self):
 6  self.client_logger = logging.getLogger("client")
 7  self.client_logger.setLevel(logging.DEBUG)
 8  h = logging.FileHandler("client.log")
 9  h.setLevel(logging.DEBUG)
10  formatter = logging.Formatter("%(asctime)s %(name)-12s 
%(levelname)-8s %(message)s")
11  h.setFormatter(formatter)
12  self.client_logger.addHandler(h)
13
14  def util(self):
15  self.client_logger.warning('This message comes from Server 
module')

$ cat -n client.py
 1  import logging
 2  import logging.handlers
 3  from server import Server
 4
 5  class Client():
 6  def __init__(self):
 7  self.client_logger = logging.getLogger("client")
 8  self.client_logger.setLevel(logging.DEBUG)
 9  h = logging.FileHandler("client.log")
10  h.setLevel(logging.DEBUG)
11  formatter = logging.Formatter("%(asctime)s %(name)-12s 
%(levelname)-8s %(message)s")
12  h.setFormatter(formatter)
13  self.client_logger.addHandler(h)
14
15  def client_test(self):
16  self.client_logger.warning("This message comes from Client 
module")
17
18  if __name__ == "__main__":
19  ser = Server()
20  cli = Client()
21  ser.util()
22  cli.client_test()
$ rm client.log ; python client.py ; cat client.log
2009-11-24 14:40:39,762 client   WARNING  This message comes from Server 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Server 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Client 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Client 
module
Googling and reading http://docs.python.org/library/logging.html didn't 
enlighten me.

Could you suggest what should I change in the above scripts so that the log 
messages would appear only once ?

Thanks,
Ron.



server.py
Description: server.py


client.py
Description: client.py


client.log
Description: client.log
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Go versus Brand X

[Antoine Pitrou]

Le Mon, 23 Nov 2009 15:30:16 -0600, Robert Kern a écrit :
> particularly constrained environments like editors that may not be 
> extensible at all.

I'm not really an expert on this, but I think most good editors /are/ 
extensible (through plugins, scripts or other things).

> You can get away with just that and have something people recognize as
> syntax highlighting, yes. But if it is possible to highlight local
> variables, globals, and types differently, that *is* useful. And you
> will even see some syntax highlighters doing more advanced things like
> that even for Python (though mostly with heuristics).

I suppose it's a matter of taste. I don't expect syntax highlighting to 
do anything else than make the source code more readable and make some 
important things stick out (comments, keywords etc.). It's probably the 
same debate as text editor vs. full IDE. Users of text editors view 
programming as a literary practice where they manipulate text, while 
users of IDEs view programming as bringing technologies together through 
specialized tools.

Interestingly, we don't know how easy to parse Go is. We just have to 
trust their word on that, but perhaps Python is easier to parse (while 
being less ugly).

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

[mk]

MRAB wrote:

In what way is it counterintuitive? In 'pythonic' the conditions are
simpler, less work is being done, therefore it's faster.


But the pythonic condition is more general: nonevar or zerovar can be 
'', 0, or None. So I thought it was more work for interpreter to compare 
those, while I thought that "is not None" is translated to one, more 
low-level and faster action. Apparently not.


As Rob pointed out (thanks):

11  31 LOAD_FAST0 (nonevar)
 34 JUMP_IF_FALSE4 (to 41)

I'm no good at py compiler or implementation internals and so I have no 
idea what bytecode "JUMP_IF_FALSE" is actually doing.


Regards,
mk


--
http://mail.python.org/mailman/listinfo/python-list

Re: xmlrpc idea for getting around the GIL

[Brian Blais]

On Nov 24, 2009, at 5:58 , sturlamolden wrote:


I have programmed parallel numerical software in Python, including on-
line signal processing. I have yet to find the GIL gets in my way.

Usually people complaining about the GIL does not know what they are
talking about.




I'd love to know which tools/libraries/approach you have found most  
profitable, and which were a waste of time.  Seems as if many people  
feel the GIL gets in the way, but perhaps they've been trying methods  
of parallelization that just aren't as effective.  What do you do?   
Do you have any sample code?



bb


--
Brian Blais
bbl...@bryant.edu
http://web.bryant.edu/~bblais



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to log messages _only once_ from all modules ?

[Soltys]

Barak, Ron pisze:

Hi,

I'm trying to add the logging module to my application, but I seem to be 
missing something.
My application (a wxPython one) has a main script that calls various helper 
classes.
I want the log messages from all modules to go to one central log file.

When I implement logging, I think that due to preparation, I get the same 
message more than once.

Here's an example:

$ cat -n server.py
 1  import logging
 2  import logging.handlers
 3
 4  class Server():
 5  def __init__(self):
 6  self.client_logger = logging.getLogger("client")
 7  self.client_logger.setLevel(logging.DEBUG)
 8  h = logging.FileHandler("client.log")
 9  h.setLevel(logging.DEBUG)
10  formatter = logging.Formatter("%(asctime)s %(name)-12s 
%(levelname)-8s %(message)s")
11  h.setFormatter(formatter)
12  self.client_logger.addHandler(h)
13
14  def util(self):
15  self.client_logger.warning('This message comes from Server 
module')

$ cat -n client.py
 1  import logging
 2  import logging.handlers
 3  from server import Server
 4
 5  class Client():
 6  def __init__(self):
 7  self.client_logger = logging.getLogger("client")
 8  self.client_logger.setLevel(logging.DEBUG)
 9  h = logging.FileHandler("client.log")
10  h.setLevel(logging.DEBUG)
11  formatter = logging.Formatter("%(asctime)s %(name)-12s 
%(levelname)-8s %(message)s")
12  h.setFormatter(formatter)
13  self.client_logger.addHandler(h)
14
15  def client_test(self):
16  self.client_logger.warning("This message comes from Client 
module")
17
18  if __name__ == "__main__":
19  ser = Server()
20  cli = Client()
21  ser.util()
22  cli.client_test()
$ rm client.log ; python client.py ; cat client.log
2009-11-24 14:40:39,762 client   WARNING  This message comes from Server 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Server 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Client 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Client 
module
Googling and reading http://docs.python.org/library/logging.html didn't 
enlighten me.

Could you suggest what should I change in the above scripts so that the log 
messages would appear only once ?

Thanks,
Ron.




Have a look at http://docs.python.org/library/logging.html#logger-objects
First thing mentioned is Logger.propagate which is, what I believe, you're
looking for ;)

--
Soltys

"Free software is a matter of liberty not price"
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to log messages _only once_ from all modules ?

[Ron Barak]

On Nov 24, 3:45 pm, Soltys  wrote:
> Barak, Ron pisze:
>
>
>
>
>
> > Hi,
>
> > I'm trying to add the logging module to my application, but I seem to be 
> > missing something.
> > My application (a wxPython one) has a main script that calls various helper 
> > classes.
> > I want the log messages from all modules to go to one central log file.
>
> > When I implement logging, I think that due to preparation, I get the same 
> > message more than once.
>
> > Here's an example:
>
> > $ cat -n server.py
> >  1  import logging
> >  2  import logging.handlers
> >  3
> >  4  class Server():
> >  5  def __init__(self):
> >  6  self.client_logger = logging.getLogger("client")
> >  7  self.client_logger.setLevel(logging.DEBUG)
> >  8  h = logging.FileHandler("client.log")
> >  9  h.setLevel(logging.DEBUG)
> > 10  formatter = logging.Formatter("%(asctime)s %(name)-12s 
> > %(levelname)-8s %(message)s")
> > 11  h.setFormatter(formatter)
> > 12  self.client_logger.addHandler(h)
> > 13
> > 14  def util(self):
> > 15  self.client_logger.warning('This message comes from Server 
> > module')
>
> > $ cat -n client.py
> >  1  import logging
> >  2  import logging.handlers
> >  3  from server import Server
> >  4
> >  5  class Client():
> >  6  def __init__(self):
> >  7  self.client_logger = logging.getLogger("client")
> >  8  self.client_logger.setLevel(logging.DEBUG)
> >  9  h = logging.FileHandler("client.log")
> > 10  h.setLevel(logging.DEBUG)
> > 11  formatter = logging.Formatter("%(asctime)s %(name)-12s 
> > %(levelname)-8s %(message)s")
> > 12  h.setFormatter(formatter)
> > 13  self.client_logger.addHandler(h)
> > 14
> > 15  def client_test(self):
> > 16  self.client_logger.warning("This message comes from Client 
> > module")
> > 17
> > 18  if __name__ == "__main__":
> > 19  ser = Server()
> > 20  cli = Client()
> > 21  ser.util()
> > 22  cli.client_test()
> > $ rm client.log ; python client.py ; cat client.log
> > 2009-11-24 14:40:39,762 client   WARNING  This message comes from 
> > Server module
> > 2009-11-24 14:40:39,762 client   WARNING  This message comes from 
> > Server module
> > 2009-11-24 14:40:39,762 client   WARNING  This message comes from 
> > Client module
> > 2009-11-24 14:40:39,762 client   WARNING  This message comes from 
> > Client module
> > Googling and readinghttp://docs.python.org/library/logging.htmldidn't 
> > enlighten me.
>
> > Could you suggest what should I change in the above scripts so that the log 
> > messages would appear only once ?
>
> > Thanks,
> > Ron.
>
> Have a look athttp://docs.python.org/library/logging.html#logger-objects
> First thing mentioned is Logger.propagate which is, what I believe, you're
> looking for ;)
>
> --
> Soltys
>
> "Free software is a matter of liberty not price"- Hide quoted text -
>
> - Show quoted text -

Hi Soltys,
I actually tried that, without any noticeable effects, viz.:

$ cat server.py
import logging
import logging.handlers

class Server():
def __init__(self):
self.client_logger = logging.getLogger("client")
self.client_logger.setLevel(logging.DEBUG)
h = logging.FileHandler("client.log")
h.setLevel(logging.DEBUG)
formatter = logging.Formatter("%(asctime)s %(name)-12s %
(levelname)-8s %(message)s")
h.setFormatter(formatter)
self.client_logger.addHandler(h)
self.client_logger.propagate = 0

def util(self):
self.client_logger.warning('This message comes from Server
module')

$ cat client.py
import logging
import logging.handlers
from server import Server

class Client():
def __init__(self):
self.client_logger = logging.getLogger("client")
self.client_logger.setLevel(logging.DEBUG)
h = logging.FileHandler("client.log")
h.setLevel(logging.DEBUG)
formatter = logging.Formatter("%(asctime)s %(name)-12s %
(levelname)-8s %(message)s")
h.setFormatter(formatter)
self.client_logger.addHandler(h)
self.client_logger.propagate = 0

def client_test(self):
self.client_logger.warning("This message comes from Client
module")

if __name__ == "__main__":
ser = Server()
cli = Client()
ser.util()
cli.client_test()

$ rm client.log ; python client.py ; cat client.log
2009-11-24 16:06:35,710 client   WARNING  This message comes from
Server module
2009-11-24 16:06:35,710 client   WARNING  This message comes from
Server module
2009-11-24 16:06:35,710 client   WARNING  This message comes from
Client module
2009-11-24 16:06:35,710 client   WARNING  This message comes from
Client module

$

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Line-continuation "Anti-Idiom" and with statement

[Neil Cerutti]

On 2009-11-23, Terry Reedy  wrote:
> Neil Cerutti wrote:
>> Unfortunately, the new "nested" with statement (which I also read
>> about today) forces me into this anti-idiom. When replacing an
>> appearance of contextlib.nested with the 3K with statement, I
>> ended up with an unexpected syntax error.
>> 
>> with (open(roster_path, 'r') as roster_file,
>>   open(disb_path, 'w') as out_file,
>>   open(report_path, 'w') as report_file):
>> 
>> The result was:
>> 
>>   File "C:\project\codxml.py", line 184
>> with (open(roster_path, 'r') as roster_file,
>>   ^
>> SyntaxError: invalid syntax
>
> Right. The first open paren is illegal.
>
> I believe that '\ \n' would always be harmless or a SyntexError
> outside of expressons. I believe 'subtly wrong' only applies
> within expressions. 
>
> So I would not call \ continuation an anti-pattern outside
> expressions. So you might suggest that the whole entry specify
> expression context to begin with. To me, your example shows why
> blanket condemnation is wrong.
>
> The HOWTOs are not scripture.

I like your suggestion. Changing the title of the anti-idiom to
"Using Backslash to Continue Expressions" seems like a good fix.
I submitted it as issue 7391.

I've done a search of the PEP's, Python-Dev, and Python-Ideas,
but I couldn't find much official discussion. The change was made
because contextlib.nested was not semantically equivalent to
actual nested with statments.

GvR noted in Python Ideas that he liked the idea of making the
new syntax parallel to the import statement sytax variant:

"import" module  ["as" name] ( ", " ["as" name] )*

So that's where the 'multi-with' statement found its model.

-- 
Neil Cerutti
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: IDE+hg

[Gerhard Häring]

Rhodri James wrote:
> On Mon, 23 Nov 2009 19:20:27 -, NiklasRTZ  wrote:
> 
>> Dear experts,
>> Since no py IDE I found has easy hg access. IDEs PIDA and Eric claim
>> Mercurial support not found i.e. buttons to clone, commit and push to
>> repositories to define dev env dvcs, editor and deployment all in 1.
> 
> I don't really understand this urge to cram everything into a single
> program, since that inevitably leads to compromises that will compromise
> just how much of Mercurial's useful and interesting functionality you
> can get at.  Still, if you really must, Emacs (and presumably vim) seems
> to be capable of working with most source control systems.

I prefer the commandline tools, too.

FWIW, Eclipse supports Mercurial through
http://www.vectrace.com/mercurialeclipse/

-- Gerhard


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

[Rob Williscroft]

mk wrote in news:mailman.923.1259070092.2873.python-l...@python.org in 
comp.lang.python:

> MRAB wrote:
>> In what way is it counterintuitive? In 'pythonic' the conditions are
>> simpler, less work is being done, therefore it's faster.
> 
> But the pythonic condition is more general: nonevar or zerovar can be 
> '', 0, or None. So I thought it was more work for interpreter to compare 
> those, while I thought that "is not None" is translated to one, more 
> low-level and faster action. Apparently not.
> 
> As Rob pointed out (thanks):
> 
> 11  31 LOAD_FAST0 (nonevar)
>   34 JUMP_IF_FALSE4 (to 41)
> 
> I'm no good at py compiler or implementation internals and so I have no 
> idea what bytecode "JUMP_IF_FALSE" is actually doing.

IIUC it implements:

http://docs.python.org/3.1/reference/expressions.html#boolean-operations

"In the context of Boolean operations, and also when expressions are used 
by control flow statements, the following values are interpreted as false: 
False, None, numeric zero of all types, and empty strings and containers 
(including strings, tuples, lists, dictionaries, sets and frozensets). All 
other values are interpreted as true. User-defined objects can customize 
their truth value by providing a __bool__() method."

In particular its implementing "... Boolean operation ... used by 
control flow ...", all in one handy op code.

Rob.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to log messages _only once_ from all modules ?

[Soltys]

Ron Barak pisze:

On Nov 24, 3:45 pm, Soltys  wrote:

Barak, Ron pisze:






Hi,
I'm trying to add the logging module to my application, but I seem to be 
missing something.
My application (a wxPython one) has a main script that calls various helper 
classes.
I want the log messages from all modules to go to one central log file.
When I implement logging, I think that due to preparation, I get the same 
message more than once.
Here's an example:
$ cat -n server.py
 1  import logging
 2  import logging.handlers
 3
 4  class Server():
 5  def __init__(self):
 6  self.client_logger = logging.getLogger("client")
 7  self.client_logger.setLevel(logging.DEBUG)
 8  h = logging.FileHandler("client.log")
 9  h.setLevel(logging.DEBUG)
10  formatter = logging.Formatter("%(asctime)s %(name)-12s 
%(levelname)-8s %(message)s")
11  h.setFormatter(formatter)
12  self.client_logger.addHandler(h)
13
14  def util(self):
15  self.client_logger.warning('This message comes from Server 
module')
$ cat -n client.py
 1  import logging
 2  import logging.handlers
 3  from server import Server
 4
 5  class Client():
 6  def __init__(self):
 7  self.client_logger = logging.getLogger("client")
 8  self.client_logger.setLevel(logging.DEBUG)
 9  h = logging.FileHandler("client.log")
10  h.setLevel(logging.DEBUG)
11  formatter = logging.Formatter("%(asctime)s %(name)-12s 
%(levelname)-8s %(message)s")
12  h.setFormatter(formatter)
13  self.client_logger.addHandler(h)
14
15  def client_test(self):
16  self.client_logger.warning("This message comes from Client 
module")
17
18  if __name__ == "__main__":
19  ser = Server()
20  cli = Client()
21  ser.util()
22  cli.client_test()
$ rm client.log ; python client.py ; cat client.log
2009-11-24 14:40:39,762 client   WARNING  This message comes from Server 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Server 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Client 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Client 
module
Googling and readinghttp://docs.python.org/library/logging.htmldidn't enlighten 
me.
Could you suggest what should I change in the above scripts so that the log 
messages would appear only once ?
Thanks,
Ron.

Have a look athttp://docs.python.org/library/logging.html#logger-objects
First thing mentioned is Logger.propagate which is, what I believe, you're
looking for ;)

--
Soltys

"Free software is a matter of liberty not price"- Hide quoted text -

- Show quoted text -


Hi Soltys,
I actually tried that, without any noticeable effects, viz.:

$ cat server.py
import logging
import logging.handlers

class Server():
def __init__(self):
self.client_logger = logging.getLogger("client")
self.client_logger.setLevel(logging.DEBUG)
h = logging.FileHandler("client.log")
h.setLevel(logging.DEBUG)
formatter = logging.Formatter("%(asctime)s %(name)-12s %
(levelname)-8s %(message)s")
h.setFormatter(formatter)
self.client_logger.addHandler(h)
self.client_logger.propagate = 0

def util(self):
self.client_logger.warning('This message comes from Server
module')

$ cat client.py
import logging
import logging.handlers
from server import Server

class Client():
def __init__(self):
self.client_logger = logging.getLogger("client")
self.client_logger.setLevel(logging.DEBUG)
h = logging.FileHandler("client.log")
h.setLevel(logging.DEBUG)
formatter = logging.Formatter("%(asctime)s %(name)-12s %
(levelname)-8s %(message)s")
h.setFormatter(formatter)
self.client_logger.addHandler(h)
self.client_logger.propagate = 0

def client_test(self):
self.client_logger.warning("This message comes from Client
module")

if __name__ == "__main__":
ser = Server()
cli = Client()
ser.util()
cli.client_test()

$ rm client.log ; python client.py ; cat client.log
2009-11-24 16:06:35,710 client   WARNING  This message comes from
Server module
2009-11-24 16:06:35,710 client   WARNING  This message comes from
Server module
2009-11-24 16:06:35,710 client   WARNING  This message comes from
Client module
2009-11-24 16:06:35,710 client   WARNING  This message comes from
Client module

$



Rename logger in server.py to server:
self.client_logger = logging.getLogger("server")

Rerun, you should get sth. like this:
$ cat client.log
2009-11-24 16:06:54,990 server   WARNING  This message comes from Server 
module
2009-11-24 16:06:54,990 client   WARNING  This message comes from Client 
module



--
Soltys

"Free software is a matter of lib

Re: pointless musings on performance

[Antoine Pitrou]

Hello,

Le Tue, 24 Nov 2009 14:41:19 +0100, mk a écrit :
> 
> As Rob pointed out (thanks):
> 
> 11  31 LOAD_FAST0 (nonevar)
>   34 JUMP_IF_FALSE4 (to 41)
> 
> I'm no good at py compiler or implementation internals and so I have no
> idea what bytecode "JUMP_IF_FALSE" is actually doing.

It tries to evaluate the op of the stack (here nonevar) in a boolean 
context (which theoretically involves calling __nonzero__ on the type) 
and then jumps if the result is False (rather than True).

You are totally right that it does /more/ than "is not None", but since 
it is executed as a single opcode rather than a sequence of several 
opcodes, the additional work it has to do is compensated (in this case) 
by the smaller overhead in bytecode interpretation.

As someone pointed out, the Python interpreter could grow CISC-like 
opcodes so as to collapse "is not None" (or generically "is not 
") into a single JUMP_IF_IS_NOT_CONST opcode. Actually, it is 
the kind of optimizations wpython does (http://code.google.com/p/
wpython/).

Regards

Antoine.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: IDE+hg

[Richard Riley]

Gerhard Häring  writes:

> Rhodri James wrote:
>> On Mon, 23 Nov 2009 19:20:27 -, NiklasRTZ  wrote:
>> 
>>> Dear experts,
>>> Since no py IDE I found has easy hg access. IDEs PIDA and Eric claim
>>> Mercurial support not found i.e. buttons to clone, commit and push to
>>> repositories to define dev env dvcs, editor and deployment all in 1.
>> 
>> I don't really understand this urge to cram everything into a single
>> program, since that inevitably leads to compromises that will
>> compromise

Huh? Cram what? Nothing is crammed into anything. The IDE/Editor is
merely programmed to hook into the external tools

>> just how much of Mercurial's useful and interesting functionality you
>> can get at.  Still, if you really must, Emacs (and presumably vim) seems
>> to be capable of working with most source control systems.
>
> I prefer the commandline tools, too.
>
> FWIW, Eclipse supports Mercurial through
> http://www.vectrace.com/mercurialeclipse/
>
> -- Gerhard

Why would you prefer the command line tools in a shell when the same
tools can be used in a way which makes navigating the output so much
easier? It strikes me as a kind of intransigence. it's a common
misconception that IDEs use their own tools all the time. They
don't. They integrate the very same tools. e.g Why the hell would I drop
to a command line to diff a file with a back version in GIT when I can
do the same in the buffer in emacs with a single hot key? Why would I
pipe the output of compile into a file then open that file when a single
hot key can fire off the SAME compiler and then list the errors in an
emacs buffer and another hot key can take me directly to the source
lines in question? Living in the past has its mements, but really.

e.g I have pylint working live in python buffers. Big time
saver. Similar with C.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to log messages _only once_ from all modules ?

[Ron Barak]

On Nov 24, 5:08 pm, Soltys  wrote:
> Ron Barak pisze:
>
>
>
>
>
> > On Nov 24, 3:45 pm, Soltys  wrote:
> >> Barak, Ron pisze:
>
> >>> Hi,
> >>> I'm trying to add the logging module to my application, but I seem to be 
> >>> missing something.
> >>> My application (a wxPython one) has a main script that calls various 
> >>> helper classes.
> >>> I want the log messages from all modules to go to one central log file.
> >>> When I implement logging, I think that due to preparation, I get the same 
> >>> message more than once.
> >>> Here's an example:
> >>> $ cat -n server.py
> >>>  1  import logging
> >>>  2  import logging.handlers
> >>>  3
> >>>  4  class Server():
> >>>  5  def __init__(self):
> >>>  6  self.client_logger = logging.getLogger("client")
> >>>  7  self.client_logger.setLevel(logging.DEBUG)
> >>>  8  h = logging.FileHandler("client.log")
> >>>  9  h.setLevel(logging.DEBUG)
> >>> 10  formatter = logging.Formatter("%(asctime)s %(name)-12s 
> >>> %(levelname)-8s %(message)s")
> >>> 11  h.setFormatter(formatter)
> >>> 12  self.client_logger.addHandler(h)
> >>> 13
> >>> 14  def util(self):
> >>> 15  self.client_logger.warning('This message comes from 
> >>> Server module')
> >>> $ cat -n client.py
> >>>  1  import logging
> >>>  2  import logging.handlers
> >>>  3  from server import Server
> >>>  4
> >>>  5  class Client():
> >>>  6  def __init__(self):
> >>>  7  self.client_logger = logging.getLogger("client")
> >>>  8  self.client_logger.setLevel(logging.DEBUG)
> >>>  9  h = logging.FileHandler("client.log")
> >>> 10  h.setLevel(logging.DEBUG)
> >>> 11  formatter = logging.Formatter("%(asctime)s %(name)-12s 
> >>> %(levelname)-8s %(message)s")
> >>> 12  h.setFormatter(formatter)
> >>> 13  self.client_logger.addHandler(h)
> >>> 14
> >>> 15  def client_test(self):
> >>> 16  self.client_logger.warning("This message comes from 
> >>> Client module")
> >>> 17
> >>> 18  if __name__ == "__main__":
> >>> 19  ser = Server()
> >>> 20  cli = Client()
> >>> 21  ser.util()
> >>> 22  cli.client_test()
> >>> $ rm client.log ; python client.py ; cat client.log
> >>> 2009-11-24 14:40:39,762 client   WARNING  This message comes from 
> >>> Server module
> >>> 2009-11-24 14:40:39,762 client   WARNING  This message comes from 
> >>> Server module
> >>> 2009-11-24 14:40:39,762 client   WARNING  This message comes from 
> >>> Client module
> >>> 2009-11-24 14:40:39,762 client   WARNING  This message comes from 
> >>> Client module
> >>> Googling and 
> >>> readinghttp://docs.python.org/library/logging.htmldidn'tenlighten me.
> >>> Could you suggest what should I change in the above scripts so that the 
> >>> log messages would appear only once ?
> >>> Thanks,
> >>> Ron.
> >> Have a look athttp://docs.python.org/library/logging.html#logger-objects
> >> First thing mentioned is Logger.propagate which is, what I believe, you're
> >> looking for ;)
>
> >> --
> >> Soltys
>
> >> "Free software is a matter of liberty not price"- Hide quoted text -
>
> >> - Show quoted text -
>
> > Hi Soltys,
> > I actually tried that, without any noticeable effects, viz.:
>
> > $ cat server.py
> > import logging
> > import logging.handlers
>
> > class Server():
> > def __init__(self):
> > self.client_logger = logging.getLogger("client")
> > self.client_logger.setLevel(logging.DEBUG)
> > h = logging.FileHandler("client.log")
> > h.setLevel(logging.DEBUG)
> > formatter = logging.Formatter("%(asctime)s %(name)-12s %
> > (levelname)-8s %(message)s")
> > h.setFormatter(formatter)
> > self.client_logger.addHandler(h)
> > self.client_logger.propagate = 0
>
> > def util(self):
> > self.client_logger.warning('This message comes from Server
> > module')
>
> > $ cat client.py
> > import logging
> > import logging.handlers
> > from server import Server
>
> > class Client():
> > def __init__(self):
> > self.client_logger = logging.getLogger("client")
> > self.client_logger.setLevel(logging.DEBUG)
> > h = logging.FileHandler("client.log")
> > h.setLevel(logging.DEBUG)
> > formatter = logging.Formatter("%(asctime)s %(name)-12s %
> > (levelname)-8s %(message)s")
> > h.setFormatter(formatter)
> > self.client_logger.addHandler(h)
> > self.client_logger.propagate = 0
>
> > def client_test(self):
> > self.client_logger.warning("This message comes from Client
> > module")
>
> > if __name__ == "__main__":
> > ser = Server()
> > cli = Client()
> > ser.util()
> > cli.client_test()
>
> > $ rm client.log ; python client.py ; cat client.log
> > 2009-11-24 16:06:35,710 client 

Re: xmlrpc idea for getting around the GIL

[Antoine Pitrou]

Le Tue, 24 Nov 2009 02:53:30 -0800, sturlamolden a écrit :
> On 22 Nov, 22:38, Patrick Stinson 
> wrote:
> 
>> Has anyone every tried wrapping the CPython lib into a daemon with an
>> RPC mechanism in order to move the GIL out of the process?
> 
>> I imagine this is how the multiprocessing module works.
> 
> It does not.

Actually, it is how multiprocessing works under Windows (for lack of the 
fork() function), except that it uses pickle by default (but it does have 
xmlrpc support).

Regards

Antoine.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Where to put the error handing test?

[Peng Yu]

On Tue, Nov 24, 2009 at 12:27 AM, alex23  wrote:
> On Nov 24, 1:15 pm, Peng Yu  wrote:
>> Suppose that I have function f() that calls g(), I can put a test on
>> the argument 'x' in either g() or f(). I'm wondering what is the
>> common practice.
>>
>> If I put the test in f(), then g() becomes more efficient when other
>> code call g() and guarantee x will pass the test even though the test
>> code in not in g(). But there might be some caller of g() that pass an
>> 'x' that might not pass the test, if there were the test in g().
>
> What you should try to do is make each function as self-contained as
> possible. f() shouldn't have to know what is a valid argument for g(),
> that's the responsibility of g(). What f() needs to know is how to
> deal with any problems that arise while using g().

This may not always be possible, because g() might call a third party
software, that I don't have the complete knowledge of. What would you
do if this case?

Another scenario:

Suppose that f_1(),...,f_(), g() are in a package, where g() is an
internal function that the end users are not suppose to call, and
f_1(),...,f_() are the functions that the end users may call.

Since all the f_1 ... f_ functions knows g(), they can be
programmed to guarantee not to pass any arguments that can not be
handled by g(). In this case, I think it is reasonable to move the
test code from g()? Is it the general accepted practice?

> As a very rough example:
>
>    def g(x):
>        try:
>            assert isinstance(x, int)
>        except AssertionError:
>            raise TypeError, "excepted int, got %s" % type(x)
>        # ... function code goes here
>
>    def f(x):
>        try:
>            g(x)
>        except TypeError:
>            # handle the problem here
>        # ... function code goes here
>
>> My thought is that if I put the test in g(x), the code of g(x) is
>> safer, but the test is not necessary when g() is called by h().
>
> This sounds strange to me. Are you stating that h() can pass values to
> g() that would be illegal for f() to pass? That sounds like a very
> dangerous design...you want each function's behaviour to be as
> consistent and predictable as it possibly can.

You misunderstood me.

h() doesn't pass any illegal arguments to g(). If I put the test code
in g(), it would be a waste of run time when h() calls g(). In this
case, and under the condition that g() is an internal function of a
package as I mentioned above, I think I should move the test code from
g() to f(). What do you think?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Beginning Question about Python functions, parameters...

[astral orange]

On Nov 23, 10:37 pm, r  wrote:
> On Nov 23, 11:19 am, astral orange <457r0...@gmail.com> wrote:
>
>
>
> > Hi, I am trying to teach myself Python and have a good book to help me
> > but I am stuck on something and I would like for someone to explain
> > the following piece of code for me and what it's actually doing.
> > Certain parts are very clear but once it enters the "def store(data,
> > full_name): " function and the "def lookup()..." function things
> > get a little confusing for me. Specifically, lines 103-108 *and* Lines
> > 110-111.
>
> > Lastly, I am not sure how to print the results I've put into this
> > program either, the book I'm reading doesn't tell me. As you can tell,
> > I am a beginner and I don't truly understand everything that is going
> > on here...a lot, but not all
>
> > Here is the code:
>
> >  92 def init(data):
> >  93     data['first'] = {}
> >  94     data['middle'] = {}
> >  95     data['last'] = {}
> >  96
> >  97 def store(data, full_name):
> >  98     names = full_name.split()
> > 100     if len(names) == 2: names.insert(1, '')
> > 101     labels = 'first', 'middle', 'last'
> > 103     for label, name in zip(labels, names):
> > 104         people = lookup(data, label, name)
> > 105     if people:
> > 106         people.append(full_name)
> > 107     else:
> > 108         data[label][name] = [full_name]
> > 109
> > 110 def lookup(data, label, name):
> > 111     return data[label].get(name)
> > 112
> > 113
> > 114 MyNames = {}
> > 115 init(MyNames)
> > 116 store(MyNames, 'John Larry Smith')
> > 117 lookup(MyNames, 'middle', 'Smith')
>
> This is a horrible example to show noobs. I think the OP could better
> understand this as a class EVEN though the OP may or may not know what
> a class *is* yet.
>
> >>> class Name():
>
>         def __init__(self, first, middle, last):
>                 self.first = first
>                 self.middle = middle
>                 self.last = last
>
> >>> name1 = Name('Guido', 'van', 'Rossum')
> >>> name1.first
> 'Guido'
> >>> name1.middle
> 'van'
> >>> name1.last
> 'Rossum'
> >>> print name1
>
> <__main__.Name instance at 0x029BFD78>
>
> This time we add a __str__ method, the result will speak louder than
> words!!
>
> >>> class Name():
>
>         def __init__(self, first, middle, last):
>                 self.first = first
>                 self.middle = middle
>                 self.last = last
>         def __str__(self):
>                 return '%s %s %s' %(self.first, self.middle, self.last)
>
> >>> name2 = Name('Terry', 'J', 'Reedy')
> >>> name2.first
> 'Terry'
> >>> name2.middle
> 'J'
> >>> name2.last
> 'Reedy'
> >>> print name2
>
> Terry J Reedy
>
> See the difference in the print statements. Now lets have some real
> fun and access each sub name by index haha!
>
> >>> class Name():
>
>         def __init__(self, first, middle, last):
>                 self.first = first
>                 self.middle = middle
>                 self.last = last
>         def __str__(self):
>                 return '%s %s %s' %(self.first, self.middle, self.last)
>         def __len__(self):
>                 return 3
>         def __getitem__(self, item):
>                 if item == 0:
>                         return self.first
>                 elif item == 1:
>                         return self.middle
>                 elif item == 2:
>                         return self.last
>                 else:
>                         raise IndexError("Index must be in range 0, 2")
>
> >>> name = Name('Joe', 'blow', 'scripter')
> >>> name[0]
> 'Joe'
> >>> name[1]
> 'blow'
> >>> name[2]
> 'scripter'
> >>> len(name)
>
> 3
>
> WOW, thats more info in a few lines than any tut i ever seen! I wish i
> could have seen that in my initial days, could have save some
> countless hours of confusion!!! Maybe i am in the wrong line of work?
>
> Should i keep going...?

Yeah, I don't think the example in the book is the best for someone
starting out. I still
am "not getting" certain parts of the program so I think I'll move on
in hopes that it will
*not* came back to haunt me and the book (along with the online
tutorial) will help me grab
more of the basics of Python programming.

As for the "class Name():" example above? Even though I haven't seen
exactly what purpose 'self' serves
yet I can follow and understand what is going on very easily, that
helps out tremendously.
Very clearly written...Thank you!

And thanks again to everyone...
-- 
http://mail.python.org/mailman/listinfo/python-list

Need help to understand a getattr syntax.

[NMarcu]

Hello all,
I need some help to understand a getattr syntax. The syntax is:

try:
getattr(self, command)(cursor, row)
except Exception, e:
print "Action failed : '%s'" % command
raise e

I don't understand why is not like this: a = getattr(), and what are
the scope of the second (): (cursor, row)

Thanks.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Need help to understand a getattr syntax.

[Diez B. Roggisch]

NMarcu wrote:

> Hello all,
> I need some help to understand a getattr syntax. The syntax is:
> 
> try:
> getattr(self, command)(cursor, row)
> except Exception, e:
> print "Action failed : '%s'" % command
> raise e
> 
> I don't understand why is not like this: a = getattr(), and what are
> the scope of the second (): (cursor, row)

The attribute in question is a callable, a method most probably. So 

 getattr(self, command)

returns a reference to the method, and the following

 (cursor, row)

calls it with cursor and row as parameters.

You could rewrite it as 

  method = getattr(self, command)
  print method
  method(cursor, row)

Diez
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

[Neil Cerutti]

On 2009-11-24, Antoine Pitrou  wrote:
> It tries to evaluate the op of the stack (here nonevar) in a
> boolean context (which theoretically involves calling
> __nonzero__ on the type) 

...or __bool__ in Py3K.

-- 
Neil Cerutti
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

[Antoine Pitrou]

Le Tue, 24 Nov 2009 15:11:29 +, Antoine Pitrou a écrit :
> Hello,
> 
> Le Tue, 24 Nov 2009 14:41:19 +0100, mk a écrit :
>> 
>> As Rob pointed out (thanks):
>> 
>> 11  31 LOAD_FAST0 (nonevar)
>>   34 JUMP_IF_FALSE4 (to 41)
>> 
>> I'm no good at py compiler or implementation internals and so I have no
>> idea what bytecode "JUMP_IF_FALSE" is actually doing.
> 
> It tries to evaluate the op of the stack (here nonevar)

I meant "the top of the stack" obviously.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: IDE+hg

[rustom]

On Nov 24, 8:13 pm, Richard Riley  wrote:
> Gerhard Häring  writes:
> > Rhodri James wrote:
> >> On Mon, 23 Nov 2009 19:20:27 -, NiklasRTZ  wrote:
>
> >>> Dear experts,
> >>> Since no py IDE I found has easy hg access. IDEs PIDA and Eric claim
> >>> Mercurial support not found i.e. buttons to clone, commit and push to
> >>> repositories to define dev env dvcs, editor and deployment all in 1.
>
> >> I don't really understand this urge to cram everything into a single
> >> program, since that inevitably leads to compromises that will
> >> compromise
>
> Huh? Cram what? Nothing is crammed into anything. The IDE/Editor is
> merely programmed to hook into the external tools
>
> >> just how much of Mercurial's useful and interesting functionality you
> >> can get at.  Still, if you really must, Emacs (and presumably vim) seems
> >> to be capable of working with most source control systems.
>
> > I prefer the commandline tools, too.
>
> > FWIW, Eclipse supports Mercurial through
> >http://www.vectrace.com/mercurialeclipse/
>
> > -- Gerhard
>
> Why would you prefer the command line tools in a shell when the same
> tools can be used in a way which makes navigating the output so much
> easier? It strikes me as a kind of intransigence. it's a common
> misconception that IDEs use their own tools all the time. They
> don't. They integrate the very same tools. e.g Why the hell would I drop
> to a command line to diff a file with a back version in GIT when I can
> do the same in the buffer in emacs with a single hot key? Why would I
> pipe the output of compile into a file then open that file when a single
> hot key can fire off the SAME compiler and then list the errors in an
> emacs buffer and another hot key can take me directly to the source
> lines in question? Living in the past has its mements, but really.
>
> e.g I have pylint working live in python buffers. Big time
> saver. Similar with C.

I sometimes think that the amount of time I spend tweaking emacs to
save my time is more than the time I spend on anything else :-)

But more seriously:
I tried to use emacs with git recently -- it was a sorry experience.
The git.el that comes with git is broken (on windows)
vc was too old for git like systems
dvc is a joke (its supposedly generic for all Distributed Version
Systems -- but everything is couched in terms of tla.
TLA! For heavens sake!
magit would not run on windows and to use egg http://github.com/bogolisk/egg
I must read magit docs.
Finally I decided to stay with what Ive used for the last 25 years --
the shell

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Where to put the error handing test?

[Peng Yu]

On Tue, Nov 24, 2009 at 4:58 AM, Dave Angel  wrote:
> Peng Yu wrote:
>>
>> On Mon, Nov 23, 2009 at 9:44 PM, Lie Ryan  wrote:
>>
>>>
>>> Peng Yu wrote:
>>>

 Suppose that I have function f() that calls g(), I can put a test on
 the argument 'x' in either g() or f(). I'm wondering what is the
 common practice.

 My thought is that if I put the test in g(x), the code of g(x) is
 safer, but the test is not necessary when g() is called by h().

 If I put the test in f(), then g() becomes more efficient when other
 code call g() and guarantee x will pass the test even though the test
 code in not in g(). But there might be some caller of g() that pass an
 'x' that might not pass the test, if there were the test in g().

>>>
>>> Typically, you test for x as early as possible, e.g. just after user
>>> input
>>> (or file or url load or whatever). After that test, you can (or should be
>>> able to) assume that all function calls will always be called with the
>>> correct argument. This is the ideal situation, it's not always easy to
>>> do.
>>>
>>> In any case though, don't optimize early.
>>>
>>
>> Let's suppose that g() is refactored out from f() and is call by not
>> only f() but other functions, and g() is likely to be called by new
>> functions.
>>
>> If I don't optimize early, I should put the test in g(), rather than f(),
>> right?
>>
>>
>
> Your question is so open-ended as to be unanswerable.  All we should do in
> this case is supply some guidelines so you can guess which one might apply
> in your particular case.
>
> You could be referring to a test that triggers alternate handling.  Or you
> could be referring to a test that notices bad input by a user, or bad data
> from an untrusted source.  Or you could be referring to a test that
> discovers bugs in your code.  And there are variations of these, depending
> on whether your user is also writing code (eval, or even import of
> user-supplied mixins), etc.
>
> The first thing that's needed in the function g() is a docstring, defining
> what inputs it expects, and what it'll do with them.  Then if it gets any
> input that doesn't meet those requirements, it might throw an exception.  Or
> it might just get an arbitrary result.  That's all up to the docstring.
>  Without any documentation, nothing is correct.
>
> Functions that are only called by trusted code need not have explicit tests
> on their inputs, since you're writing it all.  Part of debugging is catching
> those cases where f () can pass bad data to g().  If it's caused because bad
> data is passed to f(), then you have a bug in that caller.  Eventually, you
> get to the user.  If the bad data comes from the user, it should be caught
> as soon as possible, and feedback supplied right then.

I'll still confused by the guideline that an error should be caught as
early as possible.

Suppose I have the following call chain

f1() --> f2() --> f3() --> f4()

The input in f1() might cause an error in f4(). However, this error
can of cause be caught by f1(), whenever I want to do so. In the worst
case, I could duplicate the code of f2 and f3, and the test code in f4
to f1(), to catch the error in f1 rather than f4. But I don't think
that this is what you mean.

Then the problem is where to put the test code more effectively. I
would consider 'whether it is obvious to test the condition in the
give function' as the guideline. However, it might be equal obvious to
test the same thing two functions, for example, f1 and f4.

In this case, I thought originally that I should put the test code in
f1 rather than f4, if f1, f2, f3 and f4 are all the functions that I
have in the package that I am making. But it is possible that some
time later I added the function f5(),...,f10() that calls f4(). Since
f4 doesn't have the test code, f5(),...,f10() should have the same
test code. This is clearly a redundancy to the code. If I move the
test code to f4(), there is a redundancy of the code between f1 and
f4.

I'm wondering how you would solve the above problem?

> assert() ought to be the correct way to add tests in g() that test whether
> there's such a bug in f().  Unfortunately, in CPython it defaults to debug
> mode, so scripts that are run will execute those tests by default.
>  Consequently, people leave them out, to avoid slowing down code.
>
>
>
> It comes down to trust.  If you throw the code together without a test
> suite, you'll be a long time finding all the bugs in non-trivial code.  So
> add lots of defensive tests throughout the code, and pretend that's
> equivalent to a good test system.  If you're writing a library to be used by
> others, then define your public interfaces with exceptions for any invalid
> code, and write careful documentation describing what's invalid.  And if
> you're writing an end-user application, test their input as soon as you get
> it, so none of the rest of the application ever gets "invalid" data.

Having the test code for

fixing xml output format

[wadi wadi]

 Hi all,

I am creating some xml output using minidom and saving it to a file using
doc.writexml()
The output however is as follows:




bill catman




Is there a way to save xml output in a file such as xml is formatted the
right way? I mean with the right indentation and the elements values with no
new lines?



  bill catman


Note: writexml() with parameters and toprettyxml() are not giving the
desired output.

Thanks.
-- 
http://mail.python.org/mailman/listinfo/python-list

csv and mixed lists of unicode and numbers

[Sibylle Koczian]

Hello,

I want to put data from a database into a tab separated text file. This
looks like a typical application for the csv module, but there is a
snag: the rows I get from the database module (kinterbasdb in this case)
contain unicode objects and numbers. And of course the unicode objects
contain lots of non-ascii characters.

If I try to use csv.writer as is, I get UnicodeEncodeErrors. If I use
the UnicodeWriter from the module documentation, I get TypeErrors with
the numbers. (I'm using Python 2.6 - upgrading to 3.1 on this machine
would cause other complications.)

So do I have to process the rows myself and treat numbers and text
fields differently? Or what's the best way?

Here is a small example:


#!/usr/bin/env python
# -*- coding: utf-8 -*-

import csv, codecs, cStringIO
import tempfile

cData = [u'Ärger', u'Ödland', 5, u'Süßigkeit', u'élève', 6.9, u'forêt']

class UnicodeWriter:
"""
A CSV writer which will write rows to CSV file "f",
which is encoded in the given encoding.
"""

def __init__(self, f, dialect=csv.excel, encoding="utf-8", **kwds):
# Redirect output to a queue
self.queue = cStringIO.StringIO()
self.writer = csv.writer(self.queue, dialect=dialect, **kwds)
self.stream = f
self.encoder = codecs.getincrementalencoder(encoding)()

def writerow(self, row):
self.writer.writerow([s.encode("utf-8") for s in row])
# Fetch UTF-8 output from the queue ...
data = self.queue.getvalue()
data = data.decode("utf-8")
# ... and reencode it into the target encoding
data = self.encoder.encode(data)
# write to the target stream
self.stream.write(data)
# empty queue
self.queue.truncate(0)

def writerows(self, rows):
for row in rows:
self.writerow(row)

def writewithcsv(outfile, datalist):
wrt = csv.writer(outfile, dialect=csv.excel)
wrt.writerow(datalist)

def writeunicode(outfile, datalist):
wrt = UnicodeWriter(outfile)
wrt.writerow(datalist)

def main():
with tempfile.NamedTemporaryFile() as csvfile:
print "CSV file:", csvfile.name
print "Try with csv.writer"
try:
writewithcsv(csvfile, cData)
except UnicodeEncodeError as e:
print e
print "Try with UnicodeWriter"
writeunicode(csvfile, cData)
print "Ready."

if __name__ == "__main__":
main()


##

Hoping for advice,

Sibylle
--
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

[Paul Boddie]

On 24 Nov, 16:11, Antoine Pitrou  wrote:
>

[JUMP_IF_FALSE]

> It tries to evaluate the op of the stack (here nonevar) in a boolean
> context (which theoretically involves calling __nonzero__ on the type)
> and then jumps if the result is False (rather than True).

[...]

> As someone pointed out, the Python interpreter could grow CISC-like
> opcodes so as to collapse "is not None" (or generically "is not
> ") into a single JUMP_IF_IS_NOT_CONST opcode.

Of course, JUMP_IF_FALSE is already quite CISC-like, whereas testing
if something is not None could involve some fairly RISC-like
instructions: just compare the address of an operand with the address
of None. As you point out, a lot of this RISC vs. CISC analysis (and
inferences drawn from Python bytecode analysis) is somewhat academic:
the cost of the JUMP_IF_FALSE instruction is likely to be minimal in
the context of all the activity going on to evaluate the bytecodes.

I imagine that someone (or a number of people) must have profiled the
Python interpreter and shown how much time goes on the individual
bytecode implementations and how much goes on the interpreter's own
housekeeping activities. It would be interesting to see such figures.

Paul
-- 
http://mail.python.org/mailman/listinfo/python-list

howto send signal to a OS daemon on linux

[bheemesh v]

Hello,

Greetings.

I am a newbie to python programming.
Can anybody please help me with options to send a user defined signal to a
OS daemon (example a daemon snmpd).

I tried finding options to get pid of a daemon (process) by passing it's
name.
Once a pid is available thenwe can use os.kill(signal_type, signalhandler);

But how to get pid of a process by passing it's name and then how to send
signal whichis userdefind.

Kindly let me know.
Thanks in advance.

Best Regards,
Bheemesh
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: xmlrpc idea for getting around the GIL

[sturlamolden]

On 24 Nov, 16:13, Antoine Pitrou  wrote:

> >> Has anyone every tried wrapping the CPython lib into a daemon with an
> >> RPC mechanism in order to move the GIL out of the process?
>
> >> I imagine this is how the multiprocessing module works.
>
> > It does not.
>
> Actually, it is how multiprocessing works under Windows (for lack of the
> fork() function), except that it uses pickle by default (but it does have
> xmlrpc support).

Windows does not have daemons, so this is obviously incorrect. (There
are something called Windows Services, but multiprocessing does not
use them.)

Multiprocessing on Windows uses the subprocess module.











-- 
http://mail.python.org/mailman/listinfo/python-list

Re: IDE+hg

[Günther Dietrich]

NiklasRTZ  wrote:

>Since no py IDE I found has easy hg access.

Obviously, you didn't try Eclipse with PyDev () 
and Mercurial Eclipse () 
plugins.
This combination is also available stuffed into one package as 
'EasyEclipse for Python' ().

Both, pure Eclipse with plugins installed by hand, and EasyEclipse, are 
very convenient for python development.



Best regards,

Günther
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: csv and mixed lists of unicode and numbers

[Benjamin Kaplan]

On Tue, Nov 24, 2009 at 11:42 AM, Sibylle Koczian  wrote:
> Hello,
>
> I want to put data from a database into a tab separated text file. This
> looks like a typical application for the csv module, but there is a
> snag: the rows I get from the database module (kinterbasdb in this case)
> contain unicode objects and numbers. And of course the unicode objects
> contain lots of non-ascii characters.
>
> If I try to use csv.writer as is, I get UnicodeEncodeErrors. If I use
> the UnicodeWriter from the module documentation, I get TypeErrors with
> the numbers. (I'm using Python 2.6 - upgrading to 3.1 on this machine
> would cause other complications.)
>
> So do I have to process the rows myself and treat numbers and text
> fields differently? Or what's the best way?
>
> Here is a small example:
>
> 
> #!/usr/bin/env python
> # -*- coding: utf-8 -*-
>
> import csv, codecs, cStringIO
> import tempfile
>
> cData = [u'Ärger', u'Ödland', 5, u'Süßigkeit', u'élève', 6.9, u'forêt']
>
> class UnicodeWriter:
>    """
>    A CSV writer which will write rows to CSV file "f",
>    which is encoded in the given encoding.
>    """
>
>    def __init__(self, f, dialect=csv.excel, encoding="utf-8", **kwds):
>        # Redirect output to a queue
>        self.queue = cStringIO.StringIO()
>        self.writer = csv.writer(self.queue, dialect=dialect, **kwds)
>        self.stream = f
>        self.encoder = codecs.getincrementalencoder(encoding)()
>
>    def writerow(self, row):
>        self.writer.writerow([s.encode("utf-8") for s in row])

try doing [s.encode("utf-8") if isinstance(s,unicode) else s for s in row]
That way, you'll only encode the unicode strings


>        # Fetch UTF-8 output from the queue ...
>        data = self.queue.getvalue()
>        data = data.decode("utf-8")
>        # ... and reencode it into the target encoding
>        data = self.encoder.encode(data)
>        # write to the target stream
>        self.stream.write(data)
>        # empty queue
>        self.queue.truncate(0)
>
>    def writerows(self, rows):
>        for row in rows:
>            self.writerow(row)
>
> def writewithcsv(outfile, datalist):
>    wrt = csv.writer(outfile, dialect=csv.excel)
>    wrt.writerow(datalist)
>
> def writeunicode(outfile, datalist):
>    wrt = UnicodeWriter(outfile)
>    wrt.writerow(datalist)
>
> def main():
>    with tempfile.NamedTemporaryFile() as csvfile:
>        print "CSV file:", csvfile.name
>        print "Try with csv.writer"
>        try:
>            writewithcsv(csvfile, cData)
>        except UnicodeEncodeError as e:
>            print e
>        print "Try with UnicodeWriter"
>        writeunicode(csvfile, cData)
>    print "Ready."
>
> if __name__ == "__main__":
>    main()
>
>
> ##
>
> Hoping for advice,
>
> Sibylle
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Python-URL! - weekly Python news and links (Nov 24)

[Cameron Laird]

QOTW:  "... it's generally accepted that COM sucks rocks through
straws, so
explore alternatives when they're available ;-)" - Chris Withers
http://groups.google.com/group/comp.lang.python/msg/29577c851ceed167


From nothing to a complete working program - Peter Otten on
stepwise
refinement:

http://groups.google.com/group/comp.lang.python/t/f6f44b646af5b09e/8f59b2585da524a1?#8f59b2585da524a1

Handling whitespace in command line arguments:
http://groups.google.com/group/comp.lang.python/t/9a828279953b45a2/

Recognizing hex arguments in the command line:
http://groups.google.com/group/comp.lang.python/t/31d4c9386291c/

A pipeline of Python programs:
http://groups.google.com/group/comp.lang.python/t/cc06520602ae3f42/

Calling Python functions from Excel
http://groups.google.com/group/comp.lang.python/t/83aa60666c555d87/

The scope of interactive commands:   =20
http://groups.google.com/group/comp.lang.python/t/3f0d7607ed5a4a78/

List comprehensions and slice assignments - which are the
corresponding
operations for dictionaries?
http://groups.google.com/group/comp.lang.python/t/7aa443ac48f58851/

The precise semantics of [:]=20
http://groups.google.com/group/comp.lang.python/t/84b5ec30cdd26cde/

The 'with' statement doesn't allow () for implicit line
continuation:
http://comments.gmane.org/gmane.comp.python.general/645508

Grant Edwards on the best way to get help from this group :)

http://groups.google.com/group/comp.lang.python/t/b8a0c32cae495522/21e80ac383745d88?#21e80ac383745d88

Finding the root cause of slowness when sorting certain objects:
http://groups.google.com/group/comp.lang.python/t/44d80224360e085/

The fastest alternative to list.extend()
http://groups.google.com/group/comp.lang.python/t/614bfc36a09d9ab7/

A library for bijective mappings:
http://groups.google.com/group/comp.lang.python/t/785d100681f7d101/

GUI builders reviewed:
http://groups.google.com/group/comp.lang.python/t/3db5b18d77974b8/

A long thread started two weeks ago: is Python not scalable enough
for
Google?
http://groups.google.com/group/comp.lang.python/t/ceef2ae6b4472b61/



Everything Python-related you want is probably one or two clicks away
in
these pages:

Python.org's Python Language Website is the traditional
center of Pythonia
http://www.python.org
Notice especially the master FAQ
http://www.python.org/doc/FAQ.html

PythonWare complements the digest you're reading with the
marvelous daily python url
 http://www.pythonware.com/daily

Just beginning with Python?  This page is a great place to start:
http://wiki.python.org/moin/BeginnersGuide/Programmers

The Python Papers aims to publish "the efforts of Python
enthusiasts":
http://pythonpapers.org/
The Python Magazine is a technical monthly devoted to Python:
http://pythonmagazine.com

Readers have recommended the "Planet" site:
http://planet.python.org

comp.lang.python.announce announces new Python software.  Be
sure to scan this newsgroup weekly.
http://groups.google.com/group/comp.lang.python.announce/topics

Python411 indexes "podcasts ... to help people learn Python ..."
Updates appear more-than-weekly:
http://www.awaretek.com/python/index.html

The Python Package Index catalogues packages.
http://www.python.org/pypi/

Much of Python's real work takes place on Special-Interest Group
mailing lists
http://www.python.org/sigs/

Python Success Stories--from air-traffic control to on-line
match-making--can inspire you or decision-makers to whom you're
subject with a vision of what the language makes practical.
http://www.pythonology.com/success

The Python Software Foundation (PSF) has replaced the Python
Consortium as an independent nexus of activity.  It has official
responsibility for Python's development and maintenance.
http://www.python.org/psf/
Among the ways you can support PSF is with a donation.
http://www.python.org/psf/donations/

The Summary of Python Tracker Issues is an automatically generated
report summarizing new bugs, closed ones, and patch submissions.
http://search.gmane.org/?author=status%40bugs.python.org&group=gmane.com
p.python.devel&sort=date

Although unmaintained since 2002, the Cetus collection of Python
hyperlinks retains a few gems.
http://www.cetus-links.org/oo_python.html

Python FAQTS
http://python.faqts.com/

The Cookbook is a collaborative effort to capture useful and
interesting recipes.
http://code.activestate.com/recipes/langs/python/

Many Python conferences around the world are in preparation.
Watc

Re: attributes, properties, and accessors -- philosophy

[Ethan Furman]

Bruno Desthuilliers wrote:

Ethan Furman a écrit :

The problem I have with properties is my typing.  I'll end up 
assigning to an attribute, but get the spelling slightly wrong 
(capitalized, or missing an underscore -- non-obvious things when 
bug-hunting), so now I have an extra attribute which of course has 
zero effect on what I'm trying to do and I start getting wierd results 
like viewing deleted records when I *know* I set useDeleted = False... 
30 minutes later I notice it was /supposed/ to be use_deleted.  *sigh*


So -- to keep myself out of trouble -- I have started coding such 
things as, for example:


result = table.use_deleted()   # returns True or False
table.use_deleted(False)   # skip deleted records

instead of

result = table.use_deleted
table.use_deleted = False

My question:  is this [ severely | mildly | not at all ] un-pythonic?



Definitly and totally unpythonic. The first solution to your problem is 
to stick to standard naming conventions. If this is not enough, Chris 
pointed you to really useful tools. Also, you can override __setattr__ 
to catch such errors - at least during the coding/debug phase.


Good tools to know about, and a consistent naming pattern also makes 
life easier (which I have since done ;).


Let's head towards murkier waters (at least murkier to me -- hopefully 
they can be easily clarified):  some of the attributes are read-only, 
such as record count; others are not directly exposed, but still 
settable, such as table version; and still others require a small amount 
of processing... at which point do I switch from simple attribute access 
to method access?


~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: xmlrpc idea for getting around the GIL

[Antoine Pitrou]

Le Tue, 24 Nov 2009 09:27:24 -0800, sturlamolden a écrit :
> 
> Windows does not have daemons, so this is obviously incorrect. (There
> are something called Windows Services, but multiprocessing does not use
> them.)

This is nitpicking. Technically it might not be a daemon but it's used as 
such.
The important point is that it does use a (custom) form of RPC through 
marshalling, which is what the original question was about.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: UnicodeDecodeError? Argh! Nothing works! I'm tired and hurting and...

[Chris Jones]

On Tue, Nov 24, 2009 at 08:02:09AM EST, Steven D'Aprano wrote:

> Good grief, it's about six weeks away from 2010 and Thunderbird still 
> uses mbox as it's default mail box format. Hello, the nineties called, 
> they want their mail formats back! Are the tbird developers on crack or 
> something? I can't believe that they're still using that crappy format.
> 
> No, I tell a lie. I can believe it far too well.

:-)

I realize that's somewhat OT, but what mail box format do you recommend,
and why?

Thanks,

CJ
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python bijection

[Joshua Bronson]

Hey Raymond,

Thanks for your thoughtful reply! I think your idea for a class-
generation approach in the spirit of namedtuple is brilliant; looking
forward to coding this up and seeing how it feels to use it.

(By the way, it occurred to me that "bijection" is perhaps the wrong
term to use for this data structure; really it's just an injective
mapping, as it has no idea whether the function whose mappings it
contains is also surjective. (Unless we take the domain, codomain, and
range of the function being modeled to be exactly defined by the state
of the mapping at any given time. But it feels more correct to me to
interpret the mapping as a sampling of some underlying function, where
the sampling can change but the function stays the same.) So I'm
thinking of renaming the class injectivedict or idict instead of
bijection. Is that crazy?)

More responses inline:

On Nov 21, 9:22 pm, Raymond Hettinger  wrote:
> * The idea of using __call__ for looking-up inverse values was
> inspired.  That is useable, clean, and easy to remember; however, as
> discussed below, there are issues though with its actual use in real
> code.

Totally agree the call syntax has issues. Did you happen to see
Terry's suggestion to use slice syntax instead? Now *that* was
inspired. It's also much better because it works for setitem and
delitem too. I replaced the call syntax with the slice syntax on
Friday night -- would be interested to hear whether you think it's an
improvement.


> * Am not excited by the inverse iterators.  With just a regular
> mapping you can write:
>
>         for a, b in m.items() ...   # consider either a or b be the
> key and the other to be the value
>
>   That meets all of the needs that would have been served by
> iter_inverse_keys() or iter_inverse_values() or whatnot.  The mirrored
> API doesn't really provide much in the way of value added.

Hm, the one value I see the latest version of ``inverted`` adding (may
not have been in the version you saw) is that you can pass it either a
mapping, an iterable, or any object implementing an __inverted__
method. So in one case it's just syntax sugar for writing [(v, k) for
(k, v) in d.items()], but in other cases it's providing some
abstraction.



> Hope these ideas help.  The ultimate success of the Bijection code
> will depend on its clarity, simplicity, and speed.  Experiment with
> various approaches to find-out which looks the best in real code.  It
> cannot be error-prone or it is doomed.  Also, it should not introduce
> much overhead processing or else people will avoid it.  The API should
> be trivially simple so that people remember how to use it months after
> seeing it for the first time.

Thank you for the sage advice.

Best,
Josh
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: attributes, properties, and accessors -- philosophy

[Chris Rebert]

On Tue, Nov 24, 2009 at 9:39 AM, Ethan Furman  wrote:
> Bruno Desthuilliers wrote:
>> Ethan Furman a écrit :
>>> The problem I have with properties is my typing.  I'll end up assigning
>>> to an attribute, but get the spelling slightly wrong (capitalized, or
>>> missing an underscore -- non-obvious things when bug-hunting), so now I have
>>> an extra attribute which of course has zero effect on what I'm trying to do
>>> and I start getting wierd results like viewing deleted records when I *know*
>>> I set useDeleted = False... 30 minutes later I notice it was /supposed/ to
>>> be use_deleted.  *sigh*
>>>
>>> So -- to keep myself out of trouble -- I have started coding such things
>>> as, for example:
>>>
>>> result = table.use_deleted()       # returns True or False
>>> table.use_deleted(False)           # skip deleted records
>>>
>>> instead of
>>>
>>> result = table.use_deleted
>>> table.use_deleted = False
>>>
>>> My question:  is this [ severely | mildly | not at all ] un-pythonic?
>>
>>
>> Definitly and totally unpythonic. The first solution to your problem is to
>> stick to standard naming conventions. If this is not enough, Chris pointed
>> you to really useful tools. Also, you can override __setattr__ to catch such
>> errors - at least during the coding/debug phase.
>
> Good tools to know about, and a consistent naming pattern also makes life
> easier (which I have since done ;).
>
> Let's head towards murkier waters (at least murkier to me -- hopefully they
> can be easily clarified):  some of the attributes are read-only, such as
> record count; others are not directly exposed, but still settable, such as
> table version; and still others require a small amount of processing... at
> which point do I switch from simple attribute access to method access?

Thanks to the magic of properties, the end-user-programmer need not
know which you're using:

http://docs.python.org/library/functions.html#property

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

[Antoine Pitrou]

Le Tue, 24 Nov 2009 08:58:40 -0800, Paul Boddie a écrit :
> As you
> point out, a lot of this RISC vs. CISC analysis (and inferences drawn
> from Python bytecode analysis) is somewhat academic: the cost of the
> JUMP_IF_FALSE instruction is likely to be minimal in the context of all
> the activity going on to evaluate the bytecodes.

Sorry, I have trouble parsing your sentence. Do you mean bytecode 
interpretation overhead is minimal compared to the cost of actual useful 
work, or the contrary?
(IMO both are wrong by the way)

> I imagine that someone (or a number of people) must have profiled the
> Python interpreter and shown how much time goes on the individual
> bytecode implementations and how much goes on the interpreter's own
> housekeeping activities.

Well the one problem is that it's not easy to draw a line. Another 
problem is that it depends on the workload. If you are compressing large 
data or running expensive regular expressions the answer won't be the 
same as if you compute a Mandelbrot set in pure Python.

One data point is that the "computed gotos" option in py3k generally 
makes the interpreter faster by ~15%. Another data point I've heard is 
that people who have tried a very crude form of Python-to-C compilation 
(generating the exact C code corresponding to a function or method, using 
Python's C API and preserving dynamicity without attempting to be clever) 
have apparently reached speedups of up to 50% (in other words, "twice as 
fast"). So you could say that the interpretation overhead is generally 
between 15% and 50%.


-- 
http://mail.python.org/mailman/listinfo/python-list

Python-URL! - weekly Python news and links (Nov 24)

[Gabriel Genellina]

QOTW:  "... it's generally accepted that COM sucks rocks through straws, so
explore alternatives when they're available ;-)" - Chris Withers
http://groups.google.com/group/comp.lang.python/msg/29577c851ceed167


From nothing to a complete working program - Peter Otten on stepwise
refinement:

http://groups.google.com/group/comp.lang.python/t/f6f44b646af5b09e/8f59b2585da524a1?#8f59b2585da524a1

Handling whitespace in command line arguments:
http://groups.google.com/group/comp.lang.python/t/9a828279953b45a2/

Recognizing hex arguments in the command line:
http://groups.google.com/group/comp.lang.python/t/31d4c9386291c/

A pipeline of Python programs:
http://groups.google.com/group/comp.lang.python/t/cc06520602ae3f42/

Calling Python functions from Excel
http://groups.google.com/group/comp.lang.python/t/83aa60666c555d87/

The scope of interactive commands:   =20
http://groups.google.com/group/comp.lang.python/t/3f0d7607ed5a4a78/

List comprehensions and slice assignments - which are the corresponding
operations for dictionaries?
http://groups.google.com/group/comp.lang.python/t/7aa443ac48f58851/

The precise semantics of [:]=20
http://groups.google.com/group/comp.lang.python/t/84b5ec30cdd26cde/

The 'with' statement doesn't allow () for implicit line continuation:
http://comments.gmane.org/gmane.comp.python.general/645508

Grant Edwards on the best way to get help from this group :)

http://groups.google.com/group/comp.lang.python/t/b8a0c32cae495522/21e80ac383745d88?#21e80ac383745d88

Finding the root cause of slowness when sorting certain objects:
http://groups.google.com/group/comp.lang.python/t/44d80224360e085/

The fastest alternative to list.extend()
http://groups.google.com/group/comp.lang.python/t/614bfc36a09d9ab7/

A library for bijective mappings:
http://groups.google.com/group/comp.lang.python/t/785d100681f7d101/

GUI builders reviewed:
http://groups.google.com/group/comp.lang.python/t/3db5b18d77974b8/

A long thread started two weeks ago: is Python not scalable enough for
Google?
http://groups.google.com/group/comp.lang.python/t/ceef2ae6b4472b61/



Everything Python-related you want is probably one or two clicks away in
these pages:

Python.org's Python Language Website is the traditional
center of Pythonia
http://www.python.org
Notice especially the master FAQ
http://www.python.org/doc/FAQ.html

PythonWare complements the digest you're reading with the
marvelous daily python url
 http://www.pythonware.com/daily

Just beginning with Python?  This page is a great place to start:
http://wiki.python.org/moin/BeginnersGuide/Programmers

The Python Papers aims to publish "the efforts of Python enthusiasts":
http://pythonpapers.org/
The Python Magazine is a technical monthly devoted to Python:
http://pythonmagazine.com

Readers have recommended the "Planet" site:
http://planet.python.org

comp.lang.python.announce announces new Python software.  Be
sure to scan this newsgroup weekly.
http://groups.google.com/group/comp.lang.python.announce/topics

Python411 indexes "podcasts ... to help people learn Python ..."
Updates appear more-than-weekly:
http://www.awaretek.com/python/index.html

The Python Package Index catalogues packages.
http://www.python.org/pypi/

Much of Python's real work takes place on Special-Interest Group
mailing lists
http://www.python.org/sigs/

Python Success Stories--from air-traffic control to on-line
match-making--can inspire you or decision-makers to whom you're
subject with a vision of what the language makes practical.
http://www.pythonology.com/success

The Python Software Foundation (PSF) has replaced the Python
Consortium as an independent nexus of activity.  It has official
responsibility for Python's development and maintenance.
http://www.python.org/psf/
Among the ways you can support PSF is with a donation.
http://www.python.org/psf/donations/

The Summary of Python Tracker Issues is an automatically generated
report summarizing new bugs, closed ones, and patch submissions. 

http://search.gmane.org/?author=status%40bugs.python.org&group=gmane.comp.python.devel&sort=date

Although unmaintained since 2002, the Cetus collection of Python
hyperlinks retains a few gems.
http://www.cetus-links.org/oo_python.html

Python FAQTS
http://python.faqts.com/

The Cookbook is a collaborative effort to capture useful and
interesting recipes.
http://code.activestate.com/recipes/langs/python/

Re: csv and mixed lists of unicode and numbers

[Peter Otten]

Sibylle Koczian wrote:

> I want to put data from a database into a tab separated text file. This
> looks like a typical application for the csv module, but there is a
> snag: the rows I get from the database module (kinterbasdb in this case)
> contain unicode objects and numbers. And of course the unicode objects
> contain lots of non-ascii characters.
> 
> If I try to use csv.writer as is, I get UnicodeEncodeErrors. If I use
> the UnicodeWriter from the module documentation, I get TypeErrors with
> the numbers. (I'm using Python 2.6 - upgrading to 3.1 on this machine
> would cause other complications.)
> 
> So do I have to process the rows myself and treat numbers and text
> fields differently? Or what's the best way?

I'd preprocess the rows as I tend to prefer the simplest approach I can come 
up with. Example:

def recode_rows(rows, source_encoding, target_encoding):
def recode(field):
if isinstance(field, unicode):
return field.encode(target_encoding)
elif isinstance(field, str):
return unicode(field, source_encoding).encode(target_encoding)
return unicode(field).encode(target_encoding)

return (map(recode, row) for row in rows)

rows = [[1.23], [u"äöü"], [u"ÄÖÜ".encode("latin1")], [1, 2, 3]]
writer = csv.writer(sys.stdout)
writer.writerows(recode_rows(rows, "latin1", "utf-8"))

The only limitation I can see: target_encoding probably has to be a superset 
of ASCII.

Peter

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

[Tim Wintle]

On Tue, 2009-11-24 at 18:25 +, Antoine Pitrou wrote:
> Le Tue, 24 Nov 2009 08:58:40 -0800, Paul Boddie a écrit :
> > As you
> > point out, a lot of this RISC vs. CISC analysis (and inferences
> drawn
> > from Python bytecode analysis) is somewhat academic: the cost of the
> > JUMP_IF_FALSE instruction is likely to be minimal in the context of
> all the activity going on to evaluate the bytecodes.
> 
> Sorry, I have trouble parsing your sentence. Do you mean bytecode 
> interpretation overhead is minimal compared to the cost of actual
> useful work, or the contrary?
> (IMO both are wrong by the way)

Out of interest - has anyone else spotted that the call to
PyObject_IsTrue in the XXX_JUMP_IF_ blocks performs two unnecessary
pointer comparisons?

 ceval.c 
if (w == Py_True) {
Py_DECREF(w);
FAST_DISPATCH();
}
if (w == Py_False) {
Py_DECREF(w);
JUMPTO(oparg);
FAST_DISPATCH();
}
err = PyObject_IsTrue(w);
Py_DECREF(w);
.
.
.
==

 object.c 
PyObject_IsTrue(PyObject *v)
{
Py_ssize_t res;
if (v == Py_True)
return 1;
if (v == Py_False)
return 0;
.
.
.
==

Would it be worth in-lining the remaining part of PyObject_IsTrue in
ceval?

> Another data point I've heard is that people who have tried a very
> crude form of Python-to-C compilation (generating the exact C code
> corresponding to a function or method, using Python's C API and
> preserving dynamicity without attempting to be clever) have apparently
> reached speedups of up to 50% (in other words, "twice as fast").

That's roughly what I get with Cython - which does exactly that.

Tim

-- 
http://mail.python.org/mailman/listinfo/python-list

Raw strings as input from File?

[utabintarbo]

I have a log file with full Windows paths on a line. eg:
K:\A\B\C\10xx\somerandomfilename.ext->/a1/b1/c1/10xx
\somerandomfilename.ext ; txx; 11/23/2009 15:00:16 ; 1259006416

As I try to pull in the line and process it, python changes the "\10"
to a "\x08". This is before I can do anything with it. Is there a way
to specify that incoming lines (say, when using .readlines() ) should
be treated as raw strings?

TIA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: attributes, properties, and accessors -- philosophy

[Lie Ryan]

Ethan Furman wrote:


Good tools to know about, and a consistent naming pattern also makes 
life easier (which I have since done ;).


Let's head towards murkier waters (at least murkier to me -- hopefully 
they can be easily clarified):  some of the attributes are read-only, 
such as record count; others are not directly exposed, but still 
settable, such as table version; and still others require a small amount 
of processing... at which point do I switch from simple attribute access 
to method access?


~Ethan~


method accessor is not pythonic, use property

property can be read-only, write-only (!), and it can process data 
before returning and setting the real attributes.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Raw strings as input from File?

[MRAB]

utabintarbo wrote:

I have a log file with full Windows paths on a line. eg:
K:\A\B\C\10xx\somerandomfilename.ext->/a1/b1/c1/10xx
\somerandomfilename.ext ; txx; 11/23/2009 15:00:16 ; 1259006416

As I try to pull in the line and process it, python changes the "\10"
to a "\x08". This is before I can do anything with it. Is there a way
to specify that incoming lines (say, when using .readlines() ) should
be treated as raw strings?


.readlines() doesn't change the "\10" in a file to "\x08" in the string
it returns.

Could you provide some code which shows your problem?
--
http://mail.python.org/mailman/listinfo/python-list

Re: Raw strings as input from File?

[Carsten Haese]

utabintarbo wrote:
> I have a log file with full Windows paths on a line. eg:
> K:\A\B\C\10xx\somerandomfilename.ext->/a1/b1/c1/10xx
> \somerandomfilename.ext ; txx; 11/23/2009 15:00:16 ; 1259006416
> 
> As I try to pull in the line and process it, python changes the "\10"
> to a "\x08".

Python does no such thing. When Python reads bytes from a file, it
doesn't interpret or change those bytes in any way. Either there is
something else going on here that you're not telling us, or the file
doesn't contain what you think it contains. Please show us the exact
code you're using to process this file, and show us the exact contents
of the file you're processing.

--
Carsten Haese
http://informixdb.sourceforge.net

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to log messages _only once_ from all modules ?

[Vinay Sajip]

On Nov 24, 3:14 pm, Ron Barak  wrote:
> Many thanks Soltys, that did the trick (actually, no need for setting
> propagate to 0), namely,
>
[snip]

It might work for now, but instantiating a handler in an instance
constructor can be an anti-pattern. If you ever create multiple client
instances, for example, you would create multiple handlers and add
those to the logger (which is essentially persistent for the lifetime
of the process), which could result in multiple messages, corrupted
output or exceptions (in the general case).

It's generally better to declare module-level loggers via

logger = logging.getLogger(__name__)

or, if better granularity is wanted, loggers with names prefixed by
__name__ + '.' - and handler set up should typically be done in one
place in such a way as to not inadvertently create handlers multiple
times. A common pattern is to just add handlers to the root logger
(e.g. the basicConfig() API does this) - other loggers automatically
get to use them, under normal circumstances.

Regards,

Vinay Sajip
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: csv and mixed lists of unicode and numbers

[Sibylle Koczian]

Peter Otten schrieb:
> I'd preprocess the rows as I tend to prefer the simplest approach I can come 
> up with. Example:
> 
> def recode_rows(rows, source_encoding, target_encoding):
> def recode(field):
> if isinstance(field, unicode):
> return field.encode(target_encoding)
> elif isinstance(field, str):
> return unicode(field, source_encoding).encode(target_encoding)
> return unicode(field).encode(target_encoding)
> 
> return (map(recode, row) for row in rows)
> 

For this case isinstance really seems to be quite reasonable. And it was
silly of me not to think of sys.stdout as file object for the example!

> rows = [[1.23], [u"äöü"], [u"ÄÖÜ".encode("latin1")], [1, 2, 3]]
> writer = csv.writer(sys.stdout)
> writer.writerows(recode_rows(rows, "latin1", "utf-8"))
> 
> The only limitation I can see: target_encoding probably has to be a superset 
> of ASCII.
> 

Coping with umlauts and accents is quite enough for me.

This problem really goes away with Python 3 (tried it on another
machine), but something else changes too: in Python 2.6 the
documentation for the csv module explicitly says "If csvfile is a file
object, it must be opened with the ‘b’ flag on platforms where that
makes a difference." The documentation for Python 3.1 doesn't have this
sentence, and if I do that in Python 3.1 I get for all sorts of data,
even for a list with only one integer literal:

TypeError: must be bytes or buffer, not str

I don't really understand that.

Regards,
Sibylle
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Inserting Unicode text with MySQLdb in Python 2.4-2.5?

[Keith Hughitt]

Hi John,

Thanks for the suggestions: I have finally been able to get it
working :)

In anyone else runs into the same problem, here is some example code
that works in Python 2.4+:

= Begin Example ==

#!/usr/bin/env python
#-*- coding:utf-8 -*-
import sys
def main():
import MySQLdb, getpass

admin = raw_input("Database admin: ")
pw = getpass.getpass("Password: ")
db = MySQLdb.connect(use_unicode=True, charset = "utf8",
user=admin, passwd=pw)

cursor = db.cursor()
try:
cursor.execute("DROP DATABASE IF EXISTS unicode_test;")
cursor.execute("CREATE DATABASE unicode_test DEFAULT CHARACTER
SET utf8;")
except:
print ""

cursor.execute('''
CREATE TABLE `unicode_test`.`test` (
  `id`  SMALLINT unsigned NOT NULL AUTO_INCREMENT,
  `name`VARCHAR(255) NOT NULL,
   PRIMARY KEY (`id`), INDEX (`id`)
) DEFAULT CHARSET=utf8;''')

cursor.execute("INSERT INTO `unicode_test`.`test` VALUES (NULL,
'%s');" % u"Ångström")

# Test 1
print "Just printing: %s" % 'Ångström'

# Test 2
cursor.execute("SELECT name FROM unicode_test.test;")
print "From database: %s" % cursor.fetchone()[0]

# Test 3 (Manual)
print 'To verify manually: mysql -u %s -p -e "SELECT name FROM
unicode_test.test"' % admin

if __name__ == '__main__':
sys.exit(main())

= End Example 

Thanks all for taking the time to help!

Best,
Keith
-- 
http://mail.python.org/mailman/listinfo/python-list

Creating a drop down filter in Admin site

[Jase]

Hoping someone could help me out. I am updating an admin site and
added a couple "list_filter"s. They are rather long so I wanted to
turn them into a drop down filter. Can this be done? Any suggestions?

Thanks,

Jase
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Newsgroup for beginners

[bruno.desthuilli...@gmail.com]

On 20 nov, 20:42, Ethan Furman  wrote:
> Aahz wrote:
> > In article ,
> > Grant Edwards   wrote:
>
> >>You've really got to try pretty hard to create one.  But if you
> >>want to, here's how to do it:
>
> >>1) Start by complaining that your program doesn't work because
> >>   of a bug in Python.
>
> >> [...]
>
> > Post of the month!
>
> I'll second that!  I really needed a good laugh.  Many thanks!

So I'll thrice it - FWIW it indeed made it's way to the weekly python
(thanks the python-url team), but deserves much more than that. I was
so hilarious my son came out of it's room and even tried to read the
post by himself - I just wasn't able to calm down and explain him what
this was about.

Grant, if we ever meet, remind me to pay you a couple beers. Cheers !
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Where to put the error handing test?

[Lie Ryan]

Peng Yu wrote:

On Tue, Nov 24, 2009 at 4:58 AM, Dave Angel  wrote:


I'll put an extra emphasis on this:
Your question is so open-ended as to be unanswerable.  




I'll still confused by the guideline that an error should be caught as
early as possible.


but not too early. Errors must be RAISED as early as possible. Errors 
must be CAUGHT as soon as there is enough information to handle the errors.



Suppose I have the following call chain

f1() --> f2() --> f3() --> f4()

The input in f1() might cause an error in f4(). However, this error
can of cause be caught by f1(), whenever I want to do so. In the worst
case, I could duplicate the code of f2 and f3, and the test code in f4
to f1(), to catch the error in f1 rather than f4. But I don't think
that this is what you mean.


Why would f1() have faulty data? Does it come from external input? Then 
the input must be invalidated at f1(). Then f2(), f3(), and f4() does 
not require any argument checking since it is assumed that f1() calls 
them with good input.


Of course, there is cases where data validation may not be on f1. For 
example, if f1() is a function that receives keyboard input for a 
filename f1 validates whether the filename is valid. Then f2 might open 
the file and validate that the file is of the expected type (if f2 
expect a .csv file, but given an .xls file, f2 would scream out an 
error). f3 received rows/lines of data from f2 and extracts 
fields/columns from the rows; but some of the data might be faulty and 
these must be filtered out before the data is transformed by the f4. f4 
assumes f3 cleans the row and so f4 is does not do error-checking. The 
transformed data then is rewritten back by f3. Now there is an f5 which 
creates new data, the new data needs to be transformed by f4 as well and 
since f5 creates a new data, there is no way for it to create invalid 
data. Now assume f6 merges new data from another csv file; f6's data 
will also be transformed by f4, f6 can use the same validation as in f3 
but why not turn f2,f3 instead to open the other file? Now f7 will merge 
data from multiple f3 streams and transforms by f4. Then comes f8 which 
creates new data from user input, the user-inputted data will need some 
checking; and now we have trouble since the data validation is inside 
f3. But we can easily factor out the validation part into f9 and call f9 
from f3 and f8.


The example case still checks for problem as early as possible. It 
checks for problem when it is possible to determine whether a particular 
condition is problematic. The as early as possible guideline does not 
mean f1, a filename input function, must check whether the csv fields 
contains valid data. But f2, the file opening function, must be given a 
valid filename; f3, the file parser, must be given a valid file object 
with the proper type; f4, the transformer, must be given a valid row to 
transform; etc. f2 should not need to check it is given a valid 
filename, it's f1 job to validate it; f3 should not need to check 
whether the file object is of the proper type; f4 should not need to 
check it is given a valid row tuple; and so on...



Then the problem is where to put the test code more effectively. I
would consider 'whether it is obvious to test the condition in the
give function' as the guideline. However, it might be equal obvious to
test the same thing two functions, for example, f1 and f4.

In this case, I thought originally that I should put the test code in
f1 rather than f4, if f1, f2, f3 and f4 are all the functions that I
have in the package that I am making. But it is possible that some
time later I added the function f5(),...,f10() that calls f4(). Since
f4 doesn't have the test code, f5(),...,f10() should have the same
test code. This is clearly a redundancy to the code. If I move the
test code to f4(), there is a redundancy of the code between f1 and
f4.


I can't think of a good example where such redundancy would happen and 
there is no other, better way to walk around it. Could you provide a 
CONCRETE example? One that might happen, instead one that could 
theoretically happen.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Raw strings as input from File?

[utabintarbo]

On Nov 24, 3:27 pm, MRAB  wrote:
>
> .readlines() doesn't change the "\10" in a file to "\x08" in the string
> it returns.
>
> Could you provide some code which shows your problem?

Here is the code block I have so far:
for l in open(CONTENTS, 'r').readlines():
f = os.path.splitext(os.path.split(l.split('->')[0]))[0]
if f in os.listdir(DIR1) and os.path.isdir(os.path.join(DIR1,f)):
shutil.rmtree(os.path.join(DIR1,f))
if f in os.listdir(DIR2) and os.path.isdir(os.path.join(DIR2,f)):
shutil.rmtree(os.path.join(DIR2,f))

I am trying to find dirs with the basename of the initial path less
the extension in both DIR1 and DIR2

A minimally obfuscated line from the log file:
K:\sm\SMI\des\RS\Pat\10DJ\121.D5-30\1215B-B-D5-BSHOE-MM.smz->/arch_m1/
smi/des/RS/Pat/10DJ/121.D5-30\1215B-B-D5-BSHOE-MM.smz ; t9480rc ;
11/24/2009 08:16:42 ; 1259068602

What I get from the debugger/python shell:
'K:\\sm\\SMI\\des\\RS\\Pat\x08DJQ.D5-30Q5B-B-D5-BSHOE-MM.smz->/arch_m1/
smi/des/RS/Pat/10DJ/121.D5-30/1215B-B-D5-BSHOE-MM.smz ; t9480rc ;
11/24/2009 08:16:42 ; 1259068602'

TIA

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Creating a drop down filter in Admin site

[Jon Clements]

On Nov 24, 9:08 pm, Jase  wrote:
> Hoping someone could help me out. I am updating an admin site and
> added a couple "list_filter"s. They are rather long so I wanted to
> turn them into a drop down filter. Can this be done? Any suggestions?
>
> Thanks,
>
> Jase

I'm guessing you mean Django - You may well get advice here, but the
"Django Users" google group and the associated documentation of
djangoproject and how to customise the admin interface is your best
start.

hth,
Jon.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Securing a multiprocessing.BaseManager connection via SSL

[Xavier Sanz]

Hi Jonas

I've having same need so i found a solution but you need to hack a bit

I am using python 2.6.3 but i suppose it could be applicable to your
case.

First of all u need to edit /usr/lib/python2.6/lib/python2.6/
multiprocessing/connection.py

This is mine:

#
# A higher level module for using sockets (or Windows named pipes)
#
# multiprocessing/connection.py
#
# Copyright (c) 2006-2008, R Oudkerk --- see COPYING.txt
#

__all__ = [ 'Client', 'Listener', 'Pipe' ]

import os
import sys
import socket
import errno
import time
import tempfile
import itertools
# Added by Xavi
import ssl

import _multiprocessing
from multiprocessing import current_process, AuthenticationError
from multiprocessing.util import get_temp_dir, Finalize, sub_debug,
debug
from multiprocessing.forking import duplicate, close


#
#
#

BUFSIZE = 8192

_mmap_counter = itertools.count()

default_family = 'AF_INET'
families = ['AF_INET']

if hasattr(socket, 'AF_UNIX'):
default_family = 'AF_UNIX'
families += ['AF_UNIX']

if sys.platform == 'win32':
default_family = 'AF_PIPE'
families += ['AF_PIPE']

#
#
#

def arbitrary_address(family):
'''
Return an arbitrary free address for the given family
'''
if family == 'AF_INET':
return ('localhost', 0)
elif family == 'AF_UNIX':
return tempfile.mktemp(prefix='listener-', dir=get_temp_dir())
elif family == 'AF_PIPE':
return tempfile.mktemp(prefix=r'\\.\pipe\pyc-%d-%d-' %
   (os.getpid(), _mmap_counter.next()))
else:
raise ValueError('unrecognized family')


def address_type(address):
'''
Return the types of the address

This can be 'AF_INET', 'AF_UNIX', or 'AF_PIPE'
'''
if type(address) == tuple:
return 'AF_INET'
elif type(address) is str and address.startswith(''):
return 'AF_PIPE'
elif type(address) is str:
return 'AF_UNIX'
else:
raise ValueError('address type of %r unrecognized' % address)

#
# Public functions
#

class Listener(object):
'''
Returns a listener object.

This is a wrapper for a bound socket which is 'listening' for
connections, or for a Windows named pipe.
'''
def __init__(self, address=None, family=None, backlog=1,
authkey=None, sslsock=True):
family = family or (address and address_type(address)) \
 or default_family
address = address or arbitrary_address(family)

if family == 'AF_PIPE':
self._listener = PipeListener(address, backlog)
else:
self._listener = SocketListener(address, family, backlog,
sslsock)
if authkey is not None and not isinstance(authkey, bytes):
raise TypeError, 'authkey should be a byte string'

self._authkey = authkey

def accept(self):
'''
Accept a connection on the bound socket or named pipe of
`self`.

Returns a `Connection` object.
'''
c = self._listener.accept()
if self._authkey:
deliver_challenge(c, self._authkey)
answer_challenge(c, self._authkey)
return c

def close(self):
'''
Close the bound socket or named pipe of `self`.
'''
return self._listener.close()

address = property(lambda self: self._listener._address)
last_accepted = property(lambda self:
self._listener._last_accepted)


def Client(address, family=None, authkey=None, sslsock=True):
'''
Returns a connection to the address of a `Listener`
'''
family = family or address_type(address)
if family == 'AF_PIPE':
c = PipeClient(address)
else:
c = SocketClient(address,sslsock)

if authkey is not None and not isinstance(authkey, bytes):
raise TypeError, 'authkey should be a byte string'

if authkey is not None:
answer_challenge(c, authkey)
deliver_challenge(c, authkey)

return c


if sys.platform != 'win32':

def Pipe(duplex=True):
'''
Returns pair of connection objects at either end of a pipe
'''
if duplex:
s1, s2 = socket.socketpair()
c1 = _multiprocessing.Connection(os.dup(s1.fileno()))
c2 = _multiprocessing.Connection(os.dup(s2.fileno()))
s1.close()
s2.close()
else:
fd1, fd2 = os.pipe()
c1 = _multiprocessing.Connection(fd1, writable=False)
c2 = _multiprocessing.Connection(fd2, readable=False)

return c1, c2

else:

from ._multiprocessing import win32

def Pipe(duplex=True):
'''
Returns pair of connection objects at either end of a pipe
'''
address = arbitrary_address('AF_PIPE')
if duplex:
openmode = win32.PIPE_ACCESS_DUPLEX
access = win32.GENERIC_READ | win32.GENERIC_WRITE
obsize, ibsize = BUFSIZE, BUFSIZE
else:
openmode = win32.PIPE_ACCESS_INBOUND
  

Re: Raw strings as input from File?

[Jon Clements]

On Nov 24, 9:20 pm, utabintarbo  wrote:
> On Nov 24, 3:27 pm, MRAB  wrote:
>
>
>
> > .readlines() doesn't change the "\10" in a file to "\x08" in the string
> > it returns.
>
> > Could you provide some code which shows your problem?
>
> Here is the code block I have so far:
> for l in open(CONTENTS, 'r').readlines():
>     f = os.path.splitext(os.path.split(l.split('->')[0]))[0]
>     if f in os.listdir(DIR1) and os.path.isdir(os.path.join(DIR1,f)):
>         shutil.rmtree(os.path.join(DIR1,f))
>         if f in os.listdir(DIR2) and os.path.isdir(os.path.join(DIR2,f)):
>                 shutil.rmtree(os.path.join(DIR2,f))
>
> I am trying to find dirs with the basename of the initial path less
> the extension in both DIR1 and DIR2
>
> A minimally obfuscated line from the log file:
> K:\sm\SMI\des\RS\Pat\10DJ\121.D5-30\1215B-B-D5-BSHOE-MM.smz->/arch_m1/
> smi/des/RS/Pat/10DJ/121.D5-30\1215B-B-D5-BSHOE-MM.smz ; t9480rc ;
> 11/24/2009 08:16:42 ; 1259068602
>
> What I get from the debugger/python shell:
> 'K:\\sm\\SMI\\des\\RS\\Pat\x08DJQ.D5-30Q5B-B-D5-BSHOE-MM.smz->/arch_m1/
> smi/des/RS/Pat/10DJ/121.D5-30/1215B-B-D5-BSHOE-MM.smz ; t9480rc ;
> 11/24/2009 08:16:42 ; 1259068602'
>
> TIA

j...@jon-desktop:~/pytest$ cat log.txt
K:\sm\SMI\des\RS\Pat\10DJ\121.D5-30\1215B-B-D5-BSHOE-MM.smz->/arch_m1/
smi/des/RS/Pat/10DJ/121.D5-30\1215B-B-D5-BSHOE-MM.smz ; t9480rc ;
11/24/2009 08:16:42 ; 1259068602

>>> log = open('/home/jon/pytest/log.txt', 'r').readlines()
>>> log
['K:\\sm\\SMI\\des\\RS\\Pat\\10DJ\\121.D5-30\\1215B-B-D5-BSHOE-MM.smz-
>/arch_m1/\n', 'smi/des/RS/Pat/10DJ/121.D5-30\\1215B-B-D5-BSHOE-
MM.smz ; t9480rc ;\n', '11/24/2009 08:16:42 ; 1259068602\n']

See -- it's not doing anything :)

Although, "Pat\x08DJQ.D5-30Q5B-B-D5-BSHOE-MM.smz" and "Pat
\x08DJQ.D5-30Q5B-B-D5-BSHOE-MM.smz" seem to be fairly different -- are
you sure you're posting the correct output!?

Jon.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Raw strings as input from File?

[Jon Clements]

On Nov 24, 9:50 pm, Jon Clements  wrote:
> On Nov 24, 9:20 pm, utabintarbo  wrote:
[snip]
> Although, "Pat\x08DJQ.D5-30Q5B-B-D5-BSHOE-MM.smz" and "Pat
> \x08DJQ.D5-30Q5B-B-D5-BSHOE-MM.smz" seem to be fairly different -- are
> you sure you're posting the correct output!?
>

Ugh... let's try that...

Pat\10DJ\121.D5-30\1215B-B-D5-BSHOE-MM.smz
Pat\x08DJQ.D5-30Q5B-B-D5-BSHOE-MM.smz

Jon.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: csv and mixed lists of unicode and numbers

[Peter Otten]

Sibylle Koczian wrote:

> This problem really goes away with Python 3 (tried it on another
> machine), but something else changes too: in Python 2.6 the
> documentation for the csv module explicitly says "If csvfile is a file
> object, it must be opened with the ‘b’ flag on platforms where that
> makes a difference." The documentation for Python 3.1 doesn't have this
> sentence, and if I do that in Python 3.1 I get for all sorts of data,
> even for a list with only one integer literal:
> 
> TypeError: must be bytes or buffer, not str

Read the documentation for open() at

http://docs.python.org/3.1/library/functions.html#open

There are significant changes with respect to 2.x; you won't even get a file 
object anymore:

>>> open("tmp.txt", "w")
<_io.TextIOWrapper name='tmp.txt' encoding='UTF-8'>
>>> _.write("yadda")
5
>>> open("tmp.dat", "wb")
<_io.BufferedWriter name='tmp.dat'>
>>> _.write("yadda")
Traceback (most recent call last):
  File "", line 1, in 
TypeError: write() argument 1 must be bytes or buffer, not str
>>> open("tmp.dat", "wb").write(b"yadda")
5

If you specify the "b" flag in 3.x the write() method expects bytes, not 
str. The translation of newlines is now controlled by the "newline" 
argument.

Peter

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

[Benjamin Peterson]

Tim Wintle  teamrubber.com> writes:
> Out of interest - has anyone else spotted that the call to
> PyObject_IsTrue in the XXX_JUMP_IF_ blocks performs two unnecessary
> pointer comparisons?

I doubt two pointer comparisons will make much of a difference.

> Would it be worth in-lining the remaining part of PyObject_IsTrue in
> ceval?

Inlining by hand is prone to error and maintainability problems.




-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

[Terry Reedy]

Chris Rebert wrote:

On Tue, Nov 24, 2009 at 4:31 AM, Rob Williscroft  wrote:

mk wrote in news:mailman.915.1259064240.2873.python-l...@python.org in
comp.lang.python:


def pythonic():
def unpythonic():



Decidedly counterintuitive: are there special optimizations for "if
nonevar:" type of statements in cpython implementation?


from dis import dis

dis( unpythonic )

18  31 LOAD_FAST0 (nonevar)
34 LOAD_CONST   0 (None)
37 COMPARE_OP   9 (is not)
40 JUMP_IF_FALSE4 (to 47)

dis( pythonic )

11  31 LOAD_FAST0 (nonevar)
34 JUMP_IF_FALSE4 (to 41)


In other words, CPython doesn't happen to optimize `if nonevar is not
None` as much as it theoretically could (which would essentially
require a JUMP_IF_NONE opcode). Since CPython isn't known for doing
fancy optimizations, this isn't surprising.


There is a limit of 256 bytecodes. I believe there are currently fewer. 
It would seem that adding bytecodes that combine current pairs would 
speed the interpreter, depending on how often the combined pair is used. 
And people have looked for frequent pairs across a corpus of code, and 
proposed additions.


However, additional bytecodes enlarge the basic bytecode eval loop, 
possibly (and sometimes actually) leading to to more processor cache 
misses and slower execution. At least some proposed non-essential 
additions have been rejected for the reason.


Also, even though CPython-specific bytecodes are outside the language 
definition, and could theoretically be revamped every minor (x.y) 
release, there is a cost to change. Rewrite the ast to bytecode 
compiler, rewrite dis, rewrite the dis doc, and burden anyone concerned 
with multiple versions to remember. So in practice, change is minimized 
and unassigned bytecodes are left open for the future.


Optimizations that make better use of a fix set of bytecodes are a 
different topic. Guido is conservative because historically, compilier 
optimizations are too often not exactly equivalent to naive, obviously 
correct code in some overlooked corner case, leading to subtle, 
occasional bugs. Of course, byte code changes could mess-up current 
optimizations that are performed.


Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Securing a multiprocessing.BaseManager connection via SSL

[Xavier Sanz]

I recommend u to test it before use it in production environment.

On 24 nov, 22:45, Xavier Sanz  wrote:
> Hi Jonas
>
> I've having same need so i found a solution but you need to hack a bit
>
> I am using python 2.6.3 but i suppose it could be applicable to your
> case.
>
> First of all u need to edit 
> /usr/lib/python2.6/lib/python2.6/multiprocessing/connection.py
>
> This is mine:
>
> #
> # A higher level module for using sockets (or Windows named pipes)
> #
> #multiprocessing/connection.py
> #
> # Copyright (c) 2006-2008, R Oudkerk --- see COPYING.txt
> #
>
> __all__ = [ 'Client', 'Listener', 'Pipe' ]
>
> import os
> import sys
> import socket
> import errno
> import time
> import tempfile
> import itertools
> # Added by Xavi
> import ssl
>
> import _multiprocessing
> frommultiprocessingimport current_process, AuthenticationError
> frommultiprocessing.util import get_temp_dir, Finalize, sub_debug,
> debug
> frommultiprocessing.forking import duplicate, close
>
> #
> #
> #
>
> BUFSIZE = 8192
>
> _mmap_counter = itertools.count()
>
> default_family = 'AF_INET'
> families = ['AF_INET']
>
> if hasattr(socket, 'AF_UNIX'):
>     default_family = 'AF_UNIX'
>     families += ['AF_UNIX']
>
> if sys.platform == 'win32':
>     default_family = 'AF_PIPE'
>     families += ['AF_PIPE']
>
> #
> #
> #
>
> def arbitrary_address(family):
>     '''
>     Return an arbitrary free address for the given family
>     '''
>     if family == 'AF_INET':
>         return ('localhost', 0)
>     elif family == 'AF_UNIX':
>         return tempfile.mktemp(prefix='listener-', dir=get_temp_dir())
>     elif family == 'AF_PIPE':
>         return tempfile.mktemp(prefix=r'\\.\pipe\pyc-%d-%d-' %
>                                (os.getpid(), _mmap_counter.next()))
>     else:
>         raise ValueError('unrecognized family')
>
> def address_type(address):
>     '''
>     Return the types of the address
>
>     This can be 'AF_INET', 'AF_UNIX', or 'AF_PIPE'
>     '''
>     if type(address) == tuple:
>         return 'AF_INET'
>     elif type(address) is str and address.startswith(''):
>         return 'AF_PIPE'
>     elif type(address) is str:
>         return 'AF_UNIX'
>     else:
>         raise ValueError('address type of %r unrecognized' % address)
>
> #
> # Public functions
> #
>
> class Listener(object):
>     '''
>     Returns a listener object.
>
>     This is a wrapper for a bound socket which is 'listening' for
>     connections, or for a Windows named pipe.
>     '''
>     def __init__(self, address=None, family=None, backlog=1,
> authkey=None, sslsock=True):
>         family = family or (address and address_type(address)) \
>                  or default_family
>         address = address or arbitrary_address(family)
>
>         if family == 'AF_PIPE':
>             self._listener = PipeListener(address, backlog)
>         else:
>             self._listener = SocketListener(address, family, backlog,
> sslsock)
>         if authkey is not None and not isinstance(authkey, bytes):
>             raise TypeError, 'authkey should be a byte string'
>
>         self._authkey = authkey
>
>     def accept(self):
>         '''
>         Accept a connection on the bound socket or named pipe of
> `self`.
>
>         Returns a `Connection` object.
>         '''
>         c = self._listener.accept()
>         if self._authkey:
>             deliver_challenge(c, self._authkey)
>             answer_challenge(c, self._authkey)
>         return c
>
>     def close(self):
>         '''
>         Close the bound socket or named pipe of `self`.
>         '''
>         return self._listener.close()
>
>     address = property(lambda self: self._listener._address)
>     last_accepted = property(lambda self:
> self._listener._last_accepted)
>
> def Client(address, family=None, authkey=None, sslsock=True):
>     '''
>     Returns a connection to the address of a `Listener`
>     '''
>     family = family or address_type(address)
>     if family == 'AF_PIPE':
>         c = PipeClient(address)
>     else:
>         c = SocketClient(address,sslsock)
>
>     if authkey is not None and not isinstance(authkey, bytes):
>         raise TypeError, 'authkey should be a byte string'
>
>     if authkey is not None:
>         answer_challenge(c, authkey)
>         deliver_challenge(c, authkey)
>
>     return c
>
> if sys.platform != 'win32':
>
>     def Pipe(duplex=True):
>         '''
>         Returns pair of connection objects at either end of a pipe
>         '''
>         if duplex:
>             s1, s2 = socket.socketpair()
>             c1 = _multiprocessing.Connection(os.dup(s1.fileno()))
>             c2 = _multiprocessing.Connection(os.dup(s2.fileno()))
>             s1.close()
>             s2.close()
>         else:
>             fd1, fd2 = os.pipe()
>             c1 = _multiprocessing.Connection(fd1, writable=False)
>             c2 = _multiprocessing.Connection(fd2, readable=False)
>
>         return c1, c2
>
> else:
>
>     from ._multiprocessing i

Re: Beginning Question about Python functions, parameters...

[Terry Reedy]

Peter Otten wrote:

Terry Reedy wrote:



remember exactly what was stored. Maybe a typo.


That's a bug in the store() function

# as posted
def store(data, full_name):
names = full_name.split()
if len(names) == 2: names.insert(1, '')
labels = 'first', 'middle', 'last'
for label, name in zip(labels, names):
people = lookup(data, label, name)
if people:
people.append(full_name)
else:
data[label][name] = [full_name]

# what was probably intended
def store(data, full_name):
names = full_name.split()
if len(names) == 2: names.insert(1, '')
labels = 'first', 'middle', 'last'
for label, name in zip(labels, names):
people = lookup(data, label, name)
if people:
people.append(full_name)
else:
data[label][name] = [full_name]


(Note indent error, which I overlooked)
It is a nuisance when people post untested code, without identifying it 
as such. Publishing untested but trivial-to-test code like this (one 
print statement needed), especially in a book for learners, is really bad.


(I am trying to do better in the book I am working on.)

tjr

--
http://mail.python.org/mailman/listinfo/python-list

Re: Perl conversion to python...

[Hans Mulder]

J Kenneth King wrote:

Benjamin Schollnick  writes:



select(undef, undef, undef, 0.01);



# select() is a system call in Perl..
# insert Python equivalent here


That would be:

  time.sleep(0.01)

Perl has a sleep() built-in, but it truncates its argument to an int.
If you want to sleep less than a second in Perl, you have to use select
as shown.

Not so in Python.  Python's time.sleep() takes a float argument and
calls some platform-dependent function that provides sleeping with
sub-second accuracy.  On some platforms, it ends up calling the
C level select() function.

Keep in mind that in both languages, your program may end up sleeping
longer than it should due to scheduling or other activity on the system.


Hope this helps,

-- HansM

--
http://mail.python.org/mailman/listinfo/python-list

Re: UnicodeDecodeError? Argh! Nothing works! I'm tired and hurting and...

[Steven D'Aprano]

On Tue, 24 Nov 2009 13:19:10 -0500, Chris Jones wrote:

> On Tue, Nov 24, 2009 at 08:02:09AM EST, Steven D'Aprano wrote:
> 
>> Good grief, it's about six weeks away from 2010 and Thunderbird still
>> uses mbox as it's default mail box format. Hello, the nineties called,
>> they want their mail formats back! Are the tbird developers on crack or
>> something? I can't believe that they're still using that crappy format.
>> 
>> No, I tell a lie. I can believe it far too well.
> 
> :-)
> 
> I realize that's somewhat OT, but what mail box format do you recommend,
> and why?

maildir++

http://en.wikipedia.org/wiki/Maildir

Corruption is less likely, if there is corruption you'll only lose a 
single message rather than potentially everything in the mail folder[*], 
at a pinch you can read the emails using a text editor or easily grep 
through them, and compacting the mail folder is lightning fast, there's 
no wasted space in the mail folder, and there's no need to mangle lines 
starting with "From " in the body of the email.

The only major downside is that because you're dealing with potentially 
thousands of smallish files, it *may* have reduced performance on some 
older file systems that don't deal well with lots of files. These days, 
that's not a real issue.

Oh yes, and people using Windows can't use maildir because (1) it doesn't 
allow colons in names, and (2) it doesn't have atomic renames. Neither of 
these are insurmountable problems: an implementation could substitute 
another character for the colon, and while that would be a technical 
violation of the standard, it would still work. And the lack of atomic 
renames would simply mean that implementations have to be more careful 
about not having two threads writing to the one mailbox at the same time.




[*] I'm assuming normal "oops there's a bug in the mail client code" 
corruption rather than "I got drunk and started deleting random files and 
directories" corruption.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Where to put the error handing test?

[Steven D'Aprano]

On Tue, 24 Nov 2009 10:14:19 -0600, Peng Yu wrote:

> I'll still confused by the guideline that an error should be caught as
> early as possible.

I think you are confused by what that means. It means the error should be 
caught as early as possible in *time*, not in the function chain.

In Python, errors are always generated as soon as they occur. But imagine 
a system where you have a function that returns -1 on error and some 
integer on success. Then it is possible for that -1 error code to be 
passed along from one function to another to another to another before 
finally causing a fatal error, by which time you have no idea where it 
came from. That's what should be prohibited.



> Suppose I have the following call chain
> 
> f1() --> f2() --> f3() --> f4()
> 
> The input in f1() might cause an error in f4(). 

Assuming that none of the functions have side-effects, then f4 is the 
right place to catch the error. To do otherwise means that f1 needs to 
know all the possible things that can go wrong in f2, and f3, and f4.


> However, this error can
> of cause be caught by f1(), whenever I want to do so. In the worst case,
> I could duplicate the code of f2 and f3, and the test code in f4 to
> f1(), to catch the error in f1 rather than f4. But I don't think that
> this is what you mean.
> 
> Then the problem is where to put the test code more effectively. I would
> consider 'whether it is obvious to test the condition in the give
> function' as the guideline. However, it might be equal obvious to test
> the same thing two functions, for example, f1 and f4.

You're not a mathematician, are you?

For each function, define its domain: the values it must work with. 
Suppose f1 should return a result for (e.g.) all integers, but f2 only 
returns a result for *positive* integers.

Then it is very simple: f2 will raise an error if given zero or negative 
values, and so f1 must either:

* avoid that error by handling zero or negative separately; or
* catch the error and then handle zero or negative separately.

Simplified example:

def f2(x):
if x > 0:
return x+1
raise ValueError


def f1(x):  # version 1
if x > 0:
return f2(x)
else:
return "something else"

# or

def f1(x):  # version 2
try:
return f2(x)
except ValueError:
return "something else"


Version two is the preferred way to do it, since that means f1 doesn't 
need to know what the domain of f2 is. But if f2 has side-effects 
(usually a bad idea) then version one is better.


But what if f1 and f2 have the same domain?

Then it is very simple: put the error checking wherever it makes sense. 
If f2 is public, put the error checking there, and then f1 can avoid 
duplicating the error checking.

But what if f1 has the more restrictive domain? Then put the error 
checking in f1.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: csv and mixed lists of unicode and numbers

[Terry Reedy]

Sibylle Koczian wrote:

Peter Otten schrieb:
I'd preprocess the rows as I tend to prefer the simplest approach I can come 
up with. Example:


def recode_rows(rows, source_encoding, target_encoding):
def recode(field):
if isinstance(field, unicode):
return field.encode(target_encoding)
elif isinstance(field, str):
return unicode(field, source_encoding).encode(target_encoding)
return unicode(field).encode(target_encoding)

return (map(recode, row) for row in rows)



For this case isinstance really seems to be quite reasonable. And it was
silly of me not to think of sys.stdout as file object for the example!


rows = [[1.23], [u"äöü"], [u"ÄÖÜ".encode("latin1")], [1, 2, 3]]
writer = csv.writer(sys.stdout)
writer.writerows(recode_rows(rows, "latin1", "utf-8"))

The only limitation I can see: target_encoding probably has to be a superset 
of ASCII.




Coping with umlauts and accents is quite enough for me.

This problem really goes away with Python 3 (tried it on another
machine), but something else changes too: in Python 2.6 the
documentation for the csv module explicitly says "If csvfile is a file
object, it must be opened with the ‘b’ flag on platforms where that
makes a difference." The documentation for Python 3.1 doesn't have this
sentence, and if I do that in Python 3.1 I get for all sorts of data,
even for a list with only one integer literal:

TypeError: must be bytes or buffer, not str

I don't really understand that.


In Python 3, a file opened in 'b' mode is for reading and writing bytes 
with no encoding/decoding. I believe cvs works with files in text mode 
as it returns and expects strings/text for reading and writing. Perhaps 
the cvs doc should say must not be opened in 'b' mode. Not sure.


tjr

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python-URL! - weekly Python news and links (Nov 24)

[Erik Max Francis]

Bruno Desthuilliers wrote:

Cameron Laird a écrit :


Grant Edwards on the best way to get help from this group :)

http://groups.google.com/group/comp.lang.python/t/b8a0c32cae495522/21e80ac383745d88?#21e80ac383745d88


This one really deserves a POTM award !-)


It is a bit surprising to see on IRC how often people are asking for 
help but trying desperately to either 1. not provide enough information 
so that anyone could possibly help, or (even weirder) 2. argue with the 
people trying to help with them or insist they didn't really need the 
help anyway once they get the answer they're looking for.  The former is 
surely just laziness, but there's something psychological going on with 
the latter.


--
Erik Max Francis && m...@alcyone.com && http://www.alcyone.com/max/
 San Jose, CA, USA && 37 18 N 121 57 W && AIM/Y!M/Skype erikmaxfrancis
  Nine worlds I remember.
   -- Icelandic Edda of Snorri Sturluson
--
http://mail.python.org/mailman/listinfo/python-list

Re: Raw strings as input from File?

[Terry Reedy]

utabintarbo wrote:

I have a log file with full Windows paths on a line. eg:
K:\A\B\C\10xx\somerandomfilename.ext->/a1/b1/c1/10xx
\somerandomfilename.ext ; txx; 11/23/2009 15:00:16 ; 1259006416

As I try to pull in the line and process it, python changes the "\10"
to a "\x08".


This should only happen if you paste the test into your .py file as a 
string literal.



This is before I can do anything with it. Is there a way
to specify that incoming lines (say, when using .readlines() ) should
be treated as raw strings?


Or if you use execfile or compile and ask Python to interprete the input 
as code.


There are no raw strings, only raw string code literals marked with an 
'r' prefix for raw processing of the quoted text.


--
http://mail.python.org/mailman/listinfo/python-list

Re: Newsgroup for beginners

[John Machin]

On Nov 17, 2:56 pm, Grant Edwards  wrote:
> On 2009-11-17, Paul Rubin  wrote:
>
> > mrholtsr  writes:
> >> Is there a Python newsgroup for those who are strictly beginners at
> >> programming and python?
>
> > This group has its grouchy moments
>
> You've really got to try pretty hard to create one.  But if you
> want to, here's how to do it:
[snip]
>  2) Python programs are portable, so don't reveal what OS or
>     Python version you're using.  People will ask. Just ignore
>     them.

Don't supply a traceback, lest you inadvertently divulge such
information (and more!) e.g.

  File "C:\python26\lib\encodings\cp1252.py", line 15, in decode

A good safeguard against accidental disclosure of your Python version
is to avoid using the default installation folder:

File "C:\snake_teehee_ima_comedian\lib\etc_etc"

This technique, used in a question like "How can I have multiple
versions of Python installed" gives a good chance of getting a grumpy
response.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: UnicodeDecodeError? Argh! Nothing works! I'm tired and hurting and...

[samwyse]

On Nov 24, 4:43 pm, Steven D'Aprano  wrote:

> Oh yes, and people using Windows can't use maildir because (1) it doesn't
> allow colons in names, and (2) it doesn't have atomic renames. Neither of
> these are insurmountable problems: an implementation could substitute
> another character for the colon, and while that would be a technical
> violation of the standard, it would still work. And the lack of atomic
> renames would simply mean that implementations have to be more careful
> about not having two threads writing to the one mailbox at the same time.

A common work around for the former is to URL encode the names, which
let's you stick all sorts of odd characters.

I'm afraid I can't help with the latter, though.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python bijection

[Gregory Ewing]

Joshua Bronson wrote:

So I'm
thinking of renaming the class injectivedict or idict instead of
bijection. Is that crazy?)


I think you'd be better off calling it something more
down-to-earth such as bidict (bidirectional dictionary).
That way people without an advanced degree in mathematics
have a better shot at not being baffled by it.:-)

--
Greg
--
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

[Paul Boddie]

On 24 Nov, 19:25, Antoine Pitrou  wrote:
>
> Sorry, I have trouble parsing your sentence. Do you mean bytecode
> interpretation overhead is minimal compared to the cost of actual useful
> work, or the contrary?
> (IMO both are wrong by the way)

I'm referring to what you're talking about at the end. The
enhancements in Python 3 presumably came about after discussion of
"threaded interpreters", confirming that the evaluation loop in Python
2 was not exactly optimal.

> > I imagine that someone (or a number of people) must have profiled the
> > Python interpreter and shown how much time goes on the individual
> > bytecode implementations and how much goes on the interpreter's own
> > housekeeping activities.
>
> Well the one problem is that it's not easy to draw a line. Another
> problem is that it depends on the workload. If you are compressing large
> data or running expensive regular expressions the answer won't be the
> same as if you compute a Mandelbrot set in pure Python.

You need to draw the line between work done by system and external
libraries and that done by Python, but a breakdown of the time spent
executing each kind of bytecode instruction could be interesting.
Certainly, without such actual cost estimations, a simple counting of
bytecodes should hardly give an indication of how optimal some Python
code might be.

Paul
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Raw strings as input from File?

[Rhodri James]

On Tue, 24 Nov 2009 21:20:25 -, utabintarbo   
wrote:



On Nov 24, 3:27 pm, MRAB  wrote:


.readlines() doesn't change the "\10" in a file to "\x08" in the string
it returns.

Could you provide some code which shows your problem?


Here is the code block I have so far:
for l in open(CONTENTS, 'r').readlines():
f = os.path.splitext(os.path.split(l.split('->')[0]))[0]
if f in os.listdir(DIR1) and os.path.isdir(os.path.join(DIR1,f)):
shutil.rmtree(os.path.join(DIR1,f))
if f in os.listdir(DIR2) and os.path.isdir(os.path.join(DIR2,f)):
shutil.rmtree(os.path.join(DIR2,f))


Ahem.  This doesn't run.  os.path.split() returns a tuple, and calling  
os.path.splitext() doesn't work.  Given that replacing the entire loop  
contents with "print l" readily disproves your assertion, I suggest you  
cut and paste actual code if you want an answer.  Otherwise we're just  
going to keep saying "No, it doesn't", because no, it doesn't.



A minimally obfuscated line from the log file:
K:\sm\SMI\des\RS\Pat\10DJ\121.D5-30\1215B-B-D5-BSHOE-MM.smz->/arch_m1/
smi/des/RS/Pat/10DJ/121.D5-30\1215B-B-D5-BSHOE-MM.smz ; t9480rc ;
11/24/2009 08:16:42 ; 1259068602

What I get from the debugger/python shell:
'K:\\sm\\SMI\\des\\RS\\Pat\x08DJQ.D5-30Q5B-B-D5-BSHOE-MM.smz->/arch_m1/
smi/des/RS/Pat/10DJ/121.D5-30/1215B-B-D5-BSHOE-MM.smz ; t9480rc ;
11/24/2009 08:16:42 ; 1259068602'


When you do what, exactly?

--
Rhodri James *-* Wildebeest Herder to the Masses
--
http://mail.python.org/mailman/listinfo/python-list

Re: Raw strings as input from File?

[Rhodri James]

On Wed, 25 Nov 2009 01:11:29 -, Rhodri James  
 wrote:


On Tue, 24 Nov 2009 21:20:25 -, utabintarbo   
wrote:



On Nov 24, 3:27 pm, MRAB  wrote:


.readlines() doesn't change the "\10" in a file to "\x08" in the string
it returns.

Could you provide some code which shows your problem?


Here is the code block I have so far:
for l in open(CONTENTS, 'r').readlines():
f = os.path.splitext(os.path.split(l.split('->')[0]))[0]
if f in os.listdir(DIR1) and os.path.isdir(os.path.join(DIR1,f)):
shutil.rmtree(os.path.join(DIR1,f))
if f in os.listdir(DIR2) and  
os.path.isdir(os.path.join(DIR2,f)):

shutil.rmtree(os.path.join(DIR2,f))


Ahem.  This doesn't run.  os.path.split() returns a tuple, and calling  
os.path.splitext() doesn't work.


I meant, "doesn't work on a tuple".  Sigh.  It's been one of those days.

--
Rhodri James *-* Wildebeest Herder to the Masses
--
http://mail.python.org/mailman/listinfo/python-list

Re: Beginning Question about Python functions, parameters...

[r]

On Nov 24, 9:45 am, astral orange <457r0...@gmail.com> wrote:
>
> As for the "class Name():" example above? Even though I haven't seen
> exactly what purpose 'self' serves
> yet I can follow and understand what is going on very easily, that
> helps out tremendously.
> Very clearly written...Thank you!

Yes and this is the stumbling block that almost every tutorial writer
puts before the uninitiated! Simple, Simple examples are the key. The
subject matter of the example should never outshine the subject that
is being taught. Everybody understand what a first, middle, and last
name is! What really ticks me off is when some show-off tut writer
goes off using an internet protocol or mp3 tags example to explain
classes or functions... WHAT!-O

Pssst...tut writers remember this! While you may be an emacs "zen
master" playing key chords that would make most piano virtuosos
jealous, your tutorials are meant for inexperience users. Yes,
exposing your "huge intelligence" to the world may give you warm
fuzzies, but the "shock-and-awe-freak-show" will only be to the
detriment of the very ideas you are attempting to transform into these
minds.



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: attributes, properties, and accessors -- philosophy

[Ethan Furman]

Chris Rebert wrote:

On Tue, Nov 24, 2009 at 9:39 AM, Ethan Furman  wrote:


Bruno Desthuilliers wrote:


Ethan Furman a écrit :


The problem I have with properties is my typing.  I'll end up assigning
to an attribute, but get the spelling slightly wrong (capitalized, or
missing an underscore -- non-obvious things when bug-hunting), so now I have
an extra attribute which of course has zero effect on what I'm trying to do
and I start getting wierd results like viewing deleted records when I *know*
I set useDeleted = False... 30 minutes later I notice it was /supposed/ to
be use_deleted.  *sigh*

So -- to keep myself out of trouble -- I have started coding such things
as, for example:

result = table.use_deleted()   # returns True or False
table.use_deleted(False)   # skip deleted records

instead of

result = table.use_deleted
table.use_deleted = False

My question:  is this [ severely | mildly | not at all ] un-pythonic?



Definitly and totally unpythonic. The first solution to your problem is to
stick to standard naming conventions. If this is not enough, Chris pointed
you to really useful tools. Also, you can override __setattr__ to catch such
errors - at least during the coding/debug phase.


Good tools to know about, and a consistent naming pattern also makes life
easier (which I have since done ;).

Let's head towards murkier waters (at least murkier to me -- hopefully they
can be easily clarified):  some of the attributes are read-only, such as
record count; others are not directly exposed, but still settable, such as
table version; and still others require a small amount of processing... at
which point do I switch from simple attribute access to method access?



Thanks to the magic of properties, the end-user-programmer need not
know which you're using:

http://docs.python.org/library/functions.html#property


You know, when I first read that bit on properties a while back, the 
explanation of the decorators and how a property was also a decorator 
for the setter and deleter bits completely lost me.  Since then I've 
played with decorators a bit, written a configaration module that uses 
them, and just now, reading the description... it was so elegantly 
simple it almost brought tears to my eyes *sniff*.  I love Python.


Okay, I'll go back and switch all my attributes *back* to attributes -- 
and properties will be much nicer than my original implementation (using 
__getattr__ and __setattr__).


Many thanks to all who answered!

~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: python bijection

[Joshua Bronson]

On Nov 24, 6:49 pm, Gregory Ewing  wrote:
> Joshua Bronson wrote:
> > So I'm
> > thinking of renaming the class injectivedict or idict instead of
> > bijection. Is that crazy?)
>
> I think you'd be better off calling it something more
> down-to-earth such as bidict (bidirectional dictionary).
> That way people without an advanced degree in mathematics
> have a better shot at not being baffled by it.:-)
>
> --
> Greg

heh, duly noted:) bidict it is!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Raw strings as input from File?

[Grant Edwards]

On 2009-11-25, Rhodri James  wrote:
> On Tue, 24 Nov 2009 21:20:25 -, utabintarbo   
> wrote:
>
>> On Nov 24, 3:27 pm, MRAB  wrote:
>>>
>>> .readlines() doesn't change the "\10" in a file to "\x08" in the string
>>> it returns.
>>>
>>> Could you provide some code which shows your problem?
>>
>> Here is the code block I have so far:
>> for l in open(CONTENTS, 'r').readlines():
>> f = os.path.splitext(os.path.split(l.split('->')[0]))[0]
>> if f in os.listdir(DIR1) and os.path.isdir(os.path.join(DIR1,f)):
>> shutil.rmtree(os.path.join(DIR1,f))
>> if f in os.listdir(DIR2) and os.path.isdir(os.path.join(DIR2,f)):
>> shutil.rmtree(os.path.join(DIR2,f))
>
> Ahem.  This doesn't run.  os.path.split() returns a tuple, and calling  
> os.path.splitext() doesn't work.  Given that replacing the entire loop  
> contents with "print l" readily disproves your assertion, I suggest you  
> cut and paste actual code if you want an answer.  Otherwise we're just  
> going to keep saying "No, it doesn't", because no, it doesn't.

It's, um, rewarding to see my recent set of instructions being
followed.

>> A minimally obfuscated line from the log file:
>> K:\sm\SMI\des\RS\Pat\10DJ\121.D5-30\1215B-B-D5-BSHOE-MM.smz->/arch_m1/
>> smi/des/RS/Pat/10DJ/121.D5-30\1215B-B-D5-BSHOE-MM.smz ; t9480rc ;
>> 11/24/2009 08:16:42 ; 1259068602
>>
>> What I get from the debugger/python shell:
>> 'K:\\sm\\SMI\\des\\RS\\Pat\x08DJQ.D5-30Q5B-B-D5-BSHOE-MM.smz->/arch_m1/
>> smi/des/RS/Pat/10DJ/121.D5-30/1215B-B-D5-BSHOE-MM.smz ; t9480rc ;
>> 11/24/2009 08:16:42 ; 1259068602'
>
> When you do what, exactly?

;)

-- 
Grant
-- 
http://mail.python.org/mailman/listinfo/python-list

Can "self" crush itself?

[n00m]

Why does "h" instance stay alive?

class Moo:
cnt = 0
def __init__(self, x):
self.x = x
self.__class__.cnt += 1
if self.__class__.cnt > 2:
self.crush_me()
def crush_me(self):
print 'Will self be crushed?'
self = None

f = Moo(1)
g = Moo(2)
h = Moo(3)

print f
print g
print h


=== RESTART 

Will self be crushed?
<__main__.Moo instance at 0x00CC9260>
<__main__.Moo instance at 0x00CC9468>
<__main__.Moo instance at 0x00CC94B8>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: UnicodeDecodeError? Argh! Nothing works! I'm tired and hurting and...

[Chris Jones]

On Tue, Nov 24, 2009 at 05:43:32PM EST, Steven D'Aprano wrote:
> On Tue, 24 Nov 2009 13:19:10 -0500, Chris Jones wrote:
> 
> > On Tue, Nov 24, 2009 at 08:02:09AM EST, Steven D'Aprano wrote:
> > 
> >> Good grief, it's about six weeks away from 2010 and Thunderbird still
> >> uses mbox as it's default mail box format. Hello, the nineties called,
> >> they want their mail formats back! Are the tbird developers on crack or
> >> something? I can't believe that they're still using that crappy format.
> >> 
> >> No, I tell a lie. I can believe it far too well.
> > 
> > :-)
> > 
> > I realize that's somewhat OT, but what mail box format do you recommend,
> > and why?
> 
> maildir++
> 
> http://en.wikipedia.org/wiki/Maildir

Outside the two pluses, maildir also goes back to the 90s - 1995, Daniel
Berstein's orginal specification.

> Corruption is less likely, if there is corruption you'll only lose a 
> single message rather than potentially everything in the mail folder[*], 
> at a pinch you can read the emails using a text editor or easily grep 
> through them, and compacting the mail folder is lightning fast, there's 
> no wasted space in the mail folder, and there's no need to mangle lines 
> starting with "From " in the body of the email.

This last aspect very welcome.

> The only major downside is that because you're dealing with potentially 
> thousands of smallish files, it *may* have reduced performance on some 
> older file systems that don't deal well with lots of files. These days, 
> that's not a real issue.
> 
> Oh yes, and people using Windows can't use maildir because (1) it doesn't 
> allow colons in names, and (2) it doesn't have atomic renames. Neither of 
> these are insurmountable problems: an implementation could substitute 
> another character for the colon, and while that would be a technical 
> violation of the standard, it would still work. And the lack of atomic 
> renames would simply mean that implementations have to be more careful 
> about not having two threads writing to the one mailbox at the same time.
> 
> 
> [*] I'm assuming normal "oops there's a bug in the mail client code" 
> corruption rather than "I got drunk and started deleting random files and 
> directories" corruption.

I'm not concerned with the other aspects, but I'm reaching a point where
mutt is becoming rather sluggish with the mbox format, especially those
mail boxes that have more than about 3000 messages and it looks like
maildir, especially with some form of header caching might help.

Looks like running a local IMAP server would probably be more effective,
though.

Thank you for your comments.

CJ
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Can "self" crush itself?

[Ben Finney]

n00m  writes:

> def crush_me(self):
> print 'Will self be crushed?'
> self = None

As with any function, the parameter is bound to a *local* name, in this
case the name ‘self’. Whatever you rebind ‘self’ to inside the function,
the binding is lost once the function exits. None of this affects any
other bindings the same object might retain from outside the function.

It's exactly the same behaviour as this:

>>> def frobnicate(foo):
... print "Entered ‘frobnicate’"
... foo = None
... print "Leaving ‘frobnicate’"
... 
>>> bar = "syzygy"
>>> print bar
syzygy
>>> frobnicate(bar)
Entered ‘frobnicate’
Leaving ‘frobnicate’
>>> print bar
syzygy

The only difference with an instance method is how Python determines
what object to bind to the local name ‘self’. That still doesn't change
the fact that it's a local name inside that function.

-- 
 \“Read not to contradict and confute, nor to believe and take |
  `\  for granted … but to weigh and consider.” —Francis Bacon |
_o__)  |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python gui builders

[Shawn Wheatley]

It's not quite all encompassing, but I found this link last year when
looking for a similar comparison of Python GUIs:
http://ginstrom.com/scribbles/2008/02/26/python-gui-programming-platforms-for-windows/

Tkinter, Qt, GTK, IronPython... I think the only thing missing is
Jython w/ Swing or SWT. Check it out.

Shawn

On Nov 18, 5:11 pm, Stef Mientki  wrote:
> Wouldn't it be nice
> if each fan of some form of GUI-package,
> would post it's code (and resulting images) for generating one or two
> standard GUI-forms ?
-- 
http://mail.python.org/mailman/listinfo/python-list

get line number and filename in a source file

[Peng Yu]

__LINE__ __FILE__ in C++ can give the current line number and
filename. Is there a similar thing in python that can give me the
current line number and filename?

Is there a similar thing to __PRETTY_FUNCTION__ (also from C++)?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Can "self" crush itself?

[n00m]

> Whatever you rebind ‘self’ to inside the function...


Seems you are right! Thanks, Ben, for the lesson :-)


class Moo:
cnt = 0
def __init__(self, x):
self.x = x
self.__class__.cnt += 1
if self.__class__.cnt > 2:
self.crush_me()
def crush_me(self):
print 'Will self be crushed?'
self = None
print self

f = Moo(1)
g = Moo(2)
h = Moo(3)
print '='
print h



Will self be crushed?
None
=
<__main__.Moo instance at 0x00CC9468>
-- 
http://mail.python.org/mailman/listinfo/python-list