Re: Change Encoding in Py 2.5

[Stefan Behnel]

Girish, 23.07.2010 08:54:

Is ter a way to change default encoding in py 2.5 ?


Yes, but it's highly discouraged. If you tell us why you want to do it, we 
may be able to point to a better way to get what you want.


Stefan

--
http://mail.python.org/mailman/listinfo/python-list

Re: Where is the man page of python library

[Steven D'Aprano]

On Thu, 22 Jul 2010 22:24:39 -0700, march wrote:

> Steven, thank you for your reply.
> 
> It is true that I didn't read the document with enough carefulness. some
> of my questions are answered in the page I post the link of. Some are
> not. 
> But the documentation is poor. You need to read throughout the entire
> page, hoping to find what you need about one single API, and you might
> fail.

Of course the documentation can be improved. Nobody thinks it is perfect. 
But the Python documentation is actually pretty good compared to a lot of 
things out there. Perhaps you're spoilt from reading first class 
documentation and haven't had to struggle with fifteenth class 
documentation.


> I don't think "Python is a volunteer effort" can justify the poor
> documentation. Linux, glibc are based on  volunteer effort too, and they
> has good documentations.

You missed my point. Python is a volunteer effort. If you think the 
documentation is poor, what are YOU doing to fix that, apart from 
complaining? We welcome patches to the documentation.

If you don't know how to write a patch file, that's okay too. What 
suggestions do you have for fixing the docs? If people agree that your 
suggestions are good, *and* they care enough to make the effort, then 
somebody *might* generate a bug report for it, which eventually *might* 
lead to somebody producing a patch. But if you want to increase the 
chances, you need to be active and not just an armchair critic. Write the 
patch. If you can't write the patch, at least raise a bug report with a 
suggestion for fixing it.



> I am not blaming those volunteers who devote their precious time to this
> language. It will be good if the python communities get more people who
> like to write documentation.

Volunteers are welcome. The first step is to suggest an improvement to 
the docs, not just complain that they're not good enough. How would you 
fix the docs for socket.recv?



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Where is the man page of python library

[Mark Lawrence]

[Fix top posting]



On Jul 23, 12:07 pm, Steven D'Aprano  wrote:

On Thu, 22 Jul 2010 20:18:43 -0700, march wrote:

Hi, guys.



As a regular user of python, I am often annoyed by the fact that the
official python docementation is too short and too simple to satisfy my
requirement.


Python is a volunteer effort. If the docs don't suit your requirements,
we're grateful for patches.


While working with socket, I want to know every detail about every API.
I can easilly achieve that by reading man page if the language is C. But
It seems that the Python story is different.


Python is open source. Where the documentation is silent, the ultimate
authority is the source code. Particularly if the code is a thin wrapper
around the C library, which I expect (but don't know for sure) the socket
code will be.


For the interface recv(), all I got is only three sentences. "
Receive data from the socket. The return value is a string representing
the data received. The maximum amount of data to be received at once is
specified by bufsize. "
http://docs.python.org/library/socket.html#socket.socket.recv



What if the call fail?


You will get an exception, just like the page says:

 All errors raise exceptions. The normal exceptions for
 invalid argument types and out-of-memory conditions can be
 raised; errors related to socket or address semantics raise
 the error socket.error.


What if the peer close the socket?


You will get an exception, just like the Fine Manual says.


What is the
difference between blocking and non-blocking socket?


Python isn't a tutor for basic programming concepts like sockets. That's
what Google is for :)

But having said that, the documentation does explain the difference:

 In non-blocking mode, if a recv() call doesn’t find any data,
 or if a send() call can’t immediately dispose of the data,
 a error exception is raised; in blocking mode, the calls block
 until they can proceed.

http://docs.python.org/library/socket.html#socket.socket.setblocking


How could I get the errno or exception?


You get the exception the same way you get *every* exception: by catching
it with a try...except block. If you don't know that, you need to learn
the basics and not blame the documentation.

Untested, but this probably will work:

try:
 something_involving_sockets()
except socket.error, e:
 if len(e.args) == 1:
 print "Error message is:", e.args[0]
 else:
 print "errno =", e.args[0]
 print "Error message is:", e.args[1]


All the answers are "No comment".



I hate this documentation!


Don't blame the documentation for your failure to read it. It's true that
it could be improved, but most of your questions were answered by the
page you linked to.

--
Steven



On 23/07/2010 06:24, march wrote:
> Steven, thank you for your reply.
>
> It is true that I didn't read the document with enough carefulness.
> some of my questions are answered in the page I post the link of. Some
> are not.
>
> But the documentation is poor. You need to read throughout the entire
> page, hoping to find what you need about one single API, and you might
> fail.
>
> I don't think "Python is a volunteer effort" can justify the poor
> documentation. Linux, glibc are based on  volunteer effort too, and
> they has good documentations.
>
> I am not blaming those volunteers who devote their precious time to
> this language. It will be good if the python communities get more
> people who like to write documentation.
>
> Anyway, thank you again.
>

I'll be on the Python issue tracker later today. I look forward to 
seeing your first of many contributions to the "poor" Python 
documentation.  Pigs might fly? :)


Kindest regards.

Mark Lawrence.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Where is the man page of python library

[geremy condra]

On Thu, Jul 22, 2010 at 10:24 PM, march  wrote:
> Steven, thank you for your reply.
>
> It is true that I didn't read the document with enough carefulness.
> some of my questions are answered in the page I post the link of. Some
> are not.
>
> But the documentation is poor. You need to read throughout the entire
> page, hoping to find what you need about one single API, and you might
> fail.
>
> I don't think "Python is a volunteer effort" can justify the poor
> documentation. Linux, glibc are based on  volunteer effort too, and
> they has good documentations.
>
> I am not blaming those volunteers who devote their precious time to
> this language. It will be good if the python communities get more
> people who like to write documentation.
>
> Anyway, thank you again.

Two things:

1) don't top post, it's bad form.
2) I'd be happy to help you through the process of getting your doc
changes written and onto the tracker. Others more capable than myself
have offered their help once things reach that stage.

Geremy Condra
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Easy questions from a python beginner

[Steven D'Aprano]

On Thu, 22 Jul 2010 22:47:11 -0400, wheres pythonmonks wrote:

> Thanks for pointing out that swap (and my swap2) don't work everywhere
> -- is there a way to get it to work inside functions?

Not in CPython. In IronPython or Jython, maybe, I don't know enough about 
them. But even if you got it to work, it would be an implementation-
dependent trick.

[...]
> I always think that it is the language's job to express
> my thoughts...  

Ha, no, it's the language's job to execute algorithms. If it did so in a 
way that is similar to the way people think, that would be scary. Have 
you *seen* the way most people think???

*wink*


> I don't like to think that my thoughts are somehow
> constrained by the language.


Whether you "like" to think that way, or not, thoughts are influenced and 
constrained by language. While I don't accept the strong form of the 
Sapir-Whorf hypothesis (that some thoughts are *impossible* due to lack 
of language to express them, a position which has been discredited), a 
weaker form is almost certainly correct. Language influences thought.

Turing Award winner and APL creator Kenneth E. Iverson gave a lecture 
about this theme, "Notation as a tool of thought", and argued that more 
powerful notations aided thinking about computer algorithms.

Paul Graham also discusses similar ideas, such as the "blub paradox". 
Graham argues that the typical programmer is "satisfied with whatever 
language they happen to use, because it dictates the way they think about 
programs". We see this all the time, with people trying to write Java in 
Python, Perl in Python, and Ruby in Python.

And Yukihiro Matsumoto has said that one of his inspirations for creating 
Ruby was the science fiction novel Babel-17, which in turn is based on 
the Sapir-Whorf Hypothesis.



> The truth is that I don't intend to use these approaches in anything
> serious.  However, I've been known to do some metaprogramming from time
> to time.
> 
> In a recent application, I pass in a list of callables (lambdas) to be
> evaluated repeatedly.

Are you aware that lambdas are just functions? The only differences 
between a "lambda" and a function created with def is that lambda is 
syntactically limited to a single expression, and that functions created 
with lambda are anonymous (they don't have a name, or at least, not a 
meaningful name).


> Clearly, a superior solution is to pass a single lambda that returns a
> list.

I don't see why you say this is a superior solution, mostly because you 
haven't explained what the problem is.


> [Less function call dispatches] 

How? You have to generate the list at some point. Whether you do it like 
this:

functions = (sin, cos, tan)
data = (2.3, 4.5, 1.2)
result = [f(x) for f, x in zip(functions, data)]

or like this:

result = (lambda x, y, z: return (sin(x), cos(y), tan(z))
)(2.3, 4.5, 1.2)

you still end up with the same number of function calls (four). Any 
execution time will almost certainly be dominated by the work done inside 
the lambda (sin, cos and tan) rather than the infrastructure. And unless 
you have profiled your code, you would be surprised as to where the 
bottlenecks are. Your intuitions from Perl will not guide you well in 
Python -- it's a different language, and the bottlenecks are different.


> However, it might be more
> efficient to avoid the function call overhead completely and pass-in a
> string which is substituted into a string code block, compiled, and
> executed.

See, that's *exactly* what I mean about intuitions. No no no no!!! Using 
exec or eval in Python code is almost certainly a *pessimation*, not an 
optimization! I expect this will be an order of magnitude slower to 
parse, compile and execute a string than it is to execute a function. 
Using exec or friends to avoid the overhead of function calls is like 
pushing your car to work to avoid the overhead of having to get in and 
out of the car.

But of course, don't take my word for it. Write your code and profile it, 
see where the bottlenecks are. I might be wrong.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Change Encoding in Py 2.5

[Steven D'Aprano]

On Fri, 23 Jul 2010 09:18:47 +0200, Stefan Behnel wrote:

> Girish, 23.07.2010 08:54:
>> Is ter a way to change default encoding in py 2.5 ?
> 
> Yes, but it's highly discouraged. If you tell us why you want to do it,
> we may be able to point to a better way to get what you want.

I think it is discouraged because it *will* break things in the standard 
library and builtins. It's discouraged in the same way that pouring sugar 
into the petrol tank of your car is discouraged.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Where is the man page of python library

[Steven D'Aprano]

On Fri, 23 Jul 2010 08:20:03 +0100, Mark Lawrence wrote:

> [Fix top posting]

While you were fixing the top posting, did you bother to trim any 
unnecessary quoting? Let me scroll down and see...

... why no, no you didn't.

I'm not a religious man, but the verse about the mote in your brother's 
eye might be appropriate about now. It's one thing to make a point about 
another's social faux pas, and other to make an equally big one yourself 
while doing so.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

non-blocking IO EAGAIN on write

[Thomas Guettler]

Hi,

I use non-blocking io to check for timeouts. Sometimes I get EAGAIN (Resource 
temporarily unavailable)
on write(). My working code looks like this. But I am unsure how many bytes 
have been written to the
pipe if I get an EAGAIN IOError. Up to now I retry with the same chunk.

If I get EAGAIN can I just sleep, and then retry with the same data chunk?

pipe=subprocess.Popen(cmd, stdin=subprocess.PIPE, bufsize=-1)
fcntl.fcntl(pipe.stdin, fcntl.F_SETFL, os.O_NONBLOCK)

chunk_size=1024
while select.select([], [pipe.stdin], [], 5):
check_timeout()
chunk=fd.read(chunk_size)
for i_eagain in range(10):
try:
pipe.stdin.write(chunk)
except IOError, exc:
if exc.errno==errno.EAGAIN:
logging.info('write to pipe %s EAGAIN. I will try again 
i=%s. %s' % (cmd, i_eagain, exc))
time.sleep(.3)
continue
logging.error('write to pipe %s failed: %s' % (cmd, exc), 
exc_info=True)
raise
break # write was successful (default)
else:
raise Exception('Too many EAGAIN on write %s %s' % (cmd, exc), 
exc_info=True)

  Thomas


-- 
Thomas Guettler, http://www.thomas-guettler.de/
E-Mail: guettli (*) thomas-guettler + de
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Easy questions from a python beginner

[Steven D'Aprano]

On Thu, 22 Jul 2010 21:23:05 -0700, Stephen Hansen wrote:

> On 7/22/10 7:47 PM, wheres pythonmonks wrote:
[...]
>> The truth is that I don't intend to use these approaches in anything
>> serious.  However, I've been known to do some metaprogramming from time
>> to time.
> 
> Depending on how you define "metaprogramming", Python is pretty
> notoriously ill-suited towards the task (more, its basically considered
> a feature it doesn't let you go there) -- or, it allows you to do vast
> amounts of stuff with its few dark magic hooks. I've never had a
> satisfying definition of metaprogramming that more then 50% of any group
> agree with, so I'm not sure which you're looking for. :)

I disagree strongly at your characterisation that Python is notorious for 
being ill-suited towards metaprogramming. I'd say the complete opposite 
-- what is considered dark and scary metaprogramming tasks in other 
languages is considered too ordinary to even mention in Python.

If you have a class that defines a single member (or attribute in Python 
terminology) "spam", and you want to add a second "ham" to a specific 
instance, such a thing is either deep, dark metaprogramming in some 
languages, if not outright impossible. In Python it is not even 
noteworthy:

instance.ham = "something"  # *yawns*

Recently there was a thread started by some Java or C++ refugee who was 
distressed about attribute access in Python, because it made 
metaprogramming frighteningly easy:

http://mail.python.org/pipermail/python-list/2010-June/1248029.html

My response at the time was: Python makes metaprogramming *easy*:

http://mail.python.org/pipermail/python-list/2010-June/1248053.html

I can't imagine how he would have reacted if we had showed him how easy 
it is to monkey-patch built-in functions...

[...]
> But! What it doesn't let you do is get clever with syntax and write a
> sort of "simplified" or domain specific language to achieve certain
> sorts of repetitive tasks quickly. You always end up writing normal
> Python that looks like all other Python.

Exactly... 90% of the time that you think you want a DSL, Python beat you 
to it.

It is true that other languages (like Lisp, or Forth) allow you to write 
your own syntax. That's incredibly powerful, but it also has serious 
costs. If you can define your own syntax, that means every piece of code 
you look at is potentially a different language.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

make a folder to .nsi file(which finally will convert to .exe) use python

[nu]

Has any python  liberary can make a folder to .nsi file?
Thanks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Easy questions from a python beginner

[wheres pythonmonks]

Funny... just spent some time with timeit:

I wonder why I am passing in strings if the callback overhead is so light...

More funny:  it looks like inline (not passed in) lambdas can cause
python to be more efficient!
>>> import random
>>> d = [ (['A','B'][random.randint(0,1)],x,random.gauss(0,1)) for x in 
>>> xrange(0,100) ]
>>> def A1(): j = [ lambda t: (t[2]*t[1],t[2]**2+5) for t in d ]

>>> def A2(): j = [ (t[2]*t[1],t[2]**2+5) for t in d ]

>>> def A3(l): j = [ l(t) for t in d]

>>> import timeit
>>> timeit.timeit('A1()','from __main__ import A1,d',number=10);
2.2185971572472454
>>> timeit.timeit('A2()','from __main__ import A2,d',number=10);
7.2615454749912942
>>> timeit.timeit('A3(lambda t: (t[2]*t[1],t[2]**2+5))','from __main__ import 
>>> A3,d',number=10);
9.4334241349350947

So: in-line lambda possible speed improvement.  in-line tuple is slow,
passed-in callback, slowest yet?

Is this possibly right?

Hopefully someone can spot the bug?

W





On Fri, Jul 23, 2010 at 4:10 AM, Steven D'Aprano
 wrote:
> On Thu, 22 Jul 2010 22:47:11 -0400, wheres pythonmonks wrote:
>
>> Thanks for pointing out that swap (and my swap2) don't work everywhere
>> -- is there a way to get it to work inside functions?
>
> Not in CPython. In IronPython or Jython, maybe, I don't know enough about
> them. But even if you got it to work, it would be an implementation-
> dependent trick.
>
> [...]
>> I always think that it is the language's job to express
>> my thoughts...
>
> Ha, no, it's the language's job to execute algorithms. If it did so in a
> way that is similar to the way people think, that would be scary. Have
> you *seen* the way most people think???
>
> *wink*
>
>
>> I don't like to think that my thoughts are somehow
>> constrained by the language.
>
>
> Whether you "like" to think that way, or not, thoughts are influenced and
> constrained by language. While I don't accept the strong form of the
> Sapir-Whorf hypothesis (that some thoughts are *impossible* due to lack
> of language to express them, a position which has been discredited), a
> weaker form is almost certainly correct. Language influences thought.
>
> Turing Award winner and APL creator Kenneth E. Iverson gave a lecture
> about this theme, "Notation as a tool of thought", and argued that more
> powerful notations aided thinking about computer algorithms.
>
> Paul Graham also discusses similar ideas, such as the "blub paradox".
> Graham argues that the typical programmer is "satisfied with whatever
> language they happen to use, because it dictates the way they think about
> programs". We see this all the time, with people trying to write Java in
> Python, Perl in Python, and Ruby in Python.
>
> And Yukihiro Matsumoto has said that one of his inspirations for creating
> Ruby was the science fiction novel Babel-17, which in turn is based on
> the Sapir-Whorf Hypothesis.
>
>
>
>> The truth is that I don't intend to use these approaches in anything
>> serious.  However, I've been known to do some metaprogramming from time
>> to time.
>>
>> In a recent application, I pass in a list of callables (lambdas) to be
>> evaluated repeatedly.
>
> Are you aware that lambdas are just functions? The only differences
> between a "lambda" and a function created with def is that lambda is
> syntactically limited to a single expression, and that functions created
> with lambda are anonymous (they don't have a name, or at least, not a
> meaningful name).
>
>
>> Clearly, a superior solution is to pass a single lambda that returns a
>> list.
>
> I don't see why you say this is a superior solution, mostly because you
> haven't explained what the problem is.
>
>
>> [Less function call dispatches]
>
> How? You have to generate the list at some point. Whether you do it like
> this:
>
> functions = (sin, cos, tan)
> data = (2.3, 4.5, 1.2)
> result = [f(x) for f, x in zip(functions, data)]
>
> or like this:
>
> result = (lambda x, y, z: return (sin(x), cos(y), tan(z))
>    )(2.3, 4.5, 1.2)
>
> you still end up with the same number of function calls (four). Any
> execution time will almost certainly be dominated by the work done inside
> the lambda (sin, cos and tan) rather than the infrastructure. And unless
> you have profiled your code, you would be surprised as to where the
> bottlenecks are. Your intuitions from Perl will not guide you well in
> Python -- it's a different language, and the bottlenecks are different.
>
>
>> However, it might be more
>> efficient to avoid the function call overhead completely and pass-in a
>> string which is substituted into a string code block, compiled, and
>> executed.
>
> See, that's *exactly* what I mean about intuitions. No no no no!!! Using
> exec or eval in Python code is almost certainly a *pessimation*, not an
> optimization! I expect this will be an order of magnitude slower to
> parse, compile and execute a string than it is to execute a function.
> Using exec or friends to avoid the overhead of function calls is like
> push

Re: Easy questions from a python beginner

[Duncan Booth]

Steven D'Aprano  wrote:

> On Thu, 22 Jul 2010 22:47:11 -0400, wheres pythonmonks wrote:
> 
>> Thanks for pointing out that swap (and my swap2) don't work everywhere
>> -- is there a way to get it to work inside functions?

Consider languages where you can easily write a swap function (or any other 
function that updates its arguments). e.g. consider C or C#.

For C your function must take pointers to the variables, so when you call 
swap you have to make this explicit by taking the address of each variable:

   foo(&x, &y);

For C# the function takes references to the variables. Again you have to 
also make this explicit at the point of call by prefixing the argument with 
'ref':

   foo(ref x, ref y);

Python is really no different: if you want a function to rebind its 
arguments you have to make that explicit at the point of call. The only 
difference is that in Python you make the rebinding explicit by assigning 
to the names:

  x, y = foo(x, y)


-- 
Duncan Booth http://kupuguy.blogspot.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Unicode error

[dirknbr]

I am having some problems with unicode from json.

This is the error I get

UnicodeEncodeError: 'ascii' codec can't encode character u'\x93' in
position 61: ordinal not in range(128)

I have kind of developped this but obviously it's not nice, any better
ideas?

try:
text=texts[i]
text=text.encode('latin-1')
text=text.encode('utf-8')
except:
text=' '

Dirk
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Unicode error

[Steven D'Aprano]

On Fri, 23 Jul 2010 03:14:11 -0700, dirknbr wrote:

> I am having some problems with unicode from json.
> 
> This is the error I get
> 
> UnicodeEncodeError: 'ascii' codec can't encode character u'\x93' in
> position 61: ordinal not in range(128)
> 
> I have kind of developped this but obviously it's not nice, any better
> ideas?
> 
> try:
> text=texts[i]
> text=text.encode('latin-1')
> text=text.encode('utf-8')
> except:
> text=' '

Don't write bare excepts, always catch the error you want and nothing 
else. As you've written it, the result of encoding with latin-1 is thrown 
away, even if it succeeds.


text = texts[i]  # Don't hide errors here.
try:
text = text.encode('latin-1')
except UnicodeEncodeError:
try:
text = text.encode('utf-8')
except UnicodeEncodeError:
text = ' '
do_something_with(text)


Another thing you might consider is setting the error handler:

text = text.encode('utf-8', errors='ignore')

Other error handlers are 'strict' (the default), 'replace' and 
'xmlcharrefreplace'.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Unicode error

[Chris Rebert]

On Fri, Jul 23, 2010 at 3:14 AM, dirknbr  wrote:
> I am having some problems with unicode from json.
>
> This is the error I get
>
> UnicodeEncodeError: 'ascii' codec can't encode character u'\x93' in
> position 61: ordinal not in range(128)

Please include the full Traceback and the actual code that's causing
the error! We aren't mind readers.

This error basically indicates that you're incorrectly mixing byte
strings and Unicode strings somewhere.

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Unicode error

[dirknbr]

To give a bit of context. I am using twython which is a wrapper for
the JSON API

 
search=twitter.searchTwitter(s,rpp=100,page=str(it),result_type='recent',lang='en')
for u in search[u'results']:
ids.append(u[u'id'])
texts.append(u[u'text'])

This is where texts comes from.

When I then want to write texts to a file I get the unicode error.

Dirk
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Easy questions from a python beginner

[Peter Otten]

wheres pythonmonks wrote:

> Funny... just spent some time with timeit:
> 
> I wonder why I am passing in strings if the callback overhead is so
> light...
> 
> More funny:  it looks like inline (not passed in) lambdas can cause
> python to be more efficient!
 import random
 d = [ (['A','B'][random.randint(0,1)],x,random.gauss(0,1)) for x in
 xrange(0,100) ] def A1(): j = [ lambda t: (t[2]*t[1],t[2]**2+5) for
 t in d ]
> 
 def A2(): j = [ (t[2]*t[1],t[2]**2+5) for t in d ]
> 
 def A3(l): j = [ l(t) for t in d]
> 
 import timeit
 timeit.timeit('A1()','from __main__ import A1,d',number=10);
> 2.2185971572472454
 timeit.timeit('A2()','from __main__ import A2,d',number=10);
> 7.2615454749912942
 timeit.timeit('A3(lambda t: (t[2]*t[1],t[2]**2+5))','from __main__
 import A3,d',number=10);
> 9.4334241349350947
> 
> So: in-line lambda possible speed improvement.  in-line tuple is slow,
> passed-in callback, slowest yet?
> 
> Is this possibly right?
> 
> Hopefully someone can spot the bug?

A1() makes a lot of lambdas but doesn't invoke them. Once that is fixed A1() 
is indeed slower than A2():

>>> from timeit import timeit
>>> import random
>>> d = [(random.choice("AB"), x, random.gauss(0, 1)) for x in 
xrange(10**6)]
>>> def A1(d=d): return [(lambda t: (t[2]*t[1],t[2]**2+5))(t) for t in d]
...  
>>> def A2(d=d): return [(t[2]*t[1], t[2]**2+5) for t in d]   
...   
>>> assert A1() == A2()
>>> timeit("A1()", "from __main__ import A1", number=10)
14.275790929794312
>>> timeit("A2()", "from __main__ import A2", number=10)
10.31659197807312

Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Easy questions from a python beginner

[Dave Angel]

Duncan Booth wrote:


Consider languages where you can easily write a swap function (or any other 
function that updates its arguments). e.g. consider C or C#.


For C your function must take pointers to the variables, so when you call 
swap you have to make this explicit by taking the address of each variable:


   foo(&x, &y);

For C# the function takes references to the variables. Again you have to 
also make this explicit at the point of call by prefixing the argument with 
'ref':


   foo(ref x, ref y);

Python is really no different: if you want a function to rebind its 
arguments you have to make that explicit at the point of call. The only 
difference is that in Python you make the rebinding explicit by assigning 
to the names:


  x, y = foo(x, y)


  
I don't disagree with the overall point, but C has references (or at 
least C++ does, I don't think I've written any pure C code since 1992).  
If the function is declared to take references, the caller doesn't do a 
thing differently.


void foo(int &a, int &b);

is called by foo(x, y), and the function may indeed swap the arguments.

If I recall correctly, pascal is the same way.  The called function 
declares byref, and the caller doesn't do anything differently.


DaveA

--
http://mail.python.org/mailman/listinfo/python-list

Re: Easy questions from a python beginner

[Dave Angel]

wheres pythonmonks wrote:

Funny... just spent some time with timeit:

I wonder why I am passing in strings if the callback overhead is so light...

More funny:  it looks like inline (not passed in) lambdas can cause
python to be more efficient!
  

import random
d = (['A','B'][random.randint(0,1)],x,random.gauss(0,1)) for x in 
xrange(0,100) ]
def A1(): j = lambda t: (t[2]*t[1],t[2]**2+5) for t in d ]



  

def A2(): j = (t[2]*t[1],t[2]**2+5) for t in d ]



  
But A1() gives a different result.  It builds a list of function 
objects.  It doesn't actually do any of  those multiplies.  In fact, I 
don't even think it would get the same answers if you then looped 
through it, calling the functions it stored.


DaveA

--
http://mail.python.org/mailman/listinfo/python-list

Re: Visitor pattern and separating iteration

[Karsten Wutzke]

On 22 Jul., 22:25, Ian  wrote:
> Hi Karsten,
>
> On 22/07/2010 12:03, Karsten Wutzke wrote:> What is it I'm missing?
>
> I think you are making it more complicated than it really is.
>
> The visitor pattern is about bringing all the little bits that would
> otherwise be scattered all over
> many node classes into one visitor class. For code generation this means
> that all the code generation is done
> in your visitor class. For pretty-printing you have another visitor
> class. For code optimization a third
> visitor class.
>
> If there is only one sequence in which the nodes should be visited, then
> you can build it
> in to the nodes of the tree.
>
> If you need to visit your nodes in different orders you might be better
> off building a separated
> tree-walker class for each order. Thus you might have a
> post-order-walker for use with the code
> generating visitor, and a in-order-walker for a pretty-printing visitor.
>
> The optimizing walker may need to take different routes through the tree
> depending
> upon what it finds. For example, it may compute the value of the RHS of
> an assignment
> before computing the address of the LHS, so it can make better use of
> registers. Here the
> visitor class itself needs to do the walking.
>
> Regards
>
> Ian

My objects to iterate over are plain object-structures, composites. I
have only one code generation pass, so generating code and then pretty
printing is not intended/possible. I have many user options to
configure the code generation. The visitor pattern is ideal for that
matter, because I can keep the options in the generator rather than
scatter all the options into the object classes' accessor methods.
(making me think the visitor is just right and not more complicated
than needed)

I was just curious about separating the iteration code into the object
classes, because GoF said "The main reason to put the traversal
strategy in the visitor is to implement a *particular complex*
traversal, one that depends on the results of the operations on the
object structure." My code generator isn't really THAT complex (I
thought), but reviewing the latter part of the quote makes me believe
I was wrong. Code generation always depends on the previous results,
that is the children of a composite in the object structure. Now that
I realized it, I feel great about traversing in the visitor.

Python just has a few consequences that makes creating common
interfaces/abstract classes a little unusual. I'm just starting to
really love Python (or its API), though I don't feel I've already
discovered the core what it is really about.

Best regards
Karsten
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: C interpreter in Lisp/scheme/python

[francogrex]

In article <16a7e301-2e85-47eb-971e-79acc4e07...@b35g2000yqi.
googlegroups.com>, gnuist...@gmail.com says...
>This makes some sense. He replied on the newsgroup in a lengthy 
post
>that there are sufficient resources out there giving hint that 
no one
>need help me out. Then I was called "lazy" in one email and 
tersely
>given JUST the last name of an author who has many books each 
many
>100s pages, when I asked for a relevant book, as if i am a 
scholar in
>the field, although he did spend lots of words on irrelevant and
>unbeneficial things which diminished my enthusiasm. Now, I find 
out
>from you that he has/had a business concern or interest in a 
company
>that is writing/wrote lisp interpreter in C. Correct me if I am 
making
>an error. I dont want to think deprecatingly of any good soul 
but this
>is what i experienced.

No, you're not making a bad judgement. He's not the only one who 
treats newcomers with disrespect and scorn. Unfortunately many 
so-called experts in the field look down on newbies and mistreat 
them (in any programming language forum), forgetting in the 
process that they were also at a certain time newbies until 
someone gentle and nice enough teachers took the trouble to 
educate them. On the other hand there are less neurotic experts 
out there who are glad to help out someone learning. It's like in 
some universities, you have the bad "professors" who are freaks 
(probably they have a lot of problems at home, their wives 
screwing all the males on the block, daughters drug addicts etc) 
and want to take their hatred out on you, and you have the 
good and mentally stable professors who actually deserve their 
title.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Change Encoding in Py 2.5

[Christian Heimes]

> I think it is discouraged because it *will* break things in the standard 
> library and builtins. It's discouraged in the same way that pouring sugar 
> into the petrol tank of your car is discouraged.

Mythbusters have tested this urban legend. In their test the engine run
better with sugar. [1]

Christian

[1] http://mythbustersresults.com/episode15

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Change Encoding in Py 2.5

[Stefan Behnel]

Christian Heimes, 23.07.2010 15:26:

I think it is discouraged because it *will* break things in the standard
library and builtins. It's discouraged in the same way that pouring sugar
into the petrol tank of your car is discouraged.


Mythbusters have tested this urban legend. In their test the engine run
better with sugar. [1]


"Did you blow up your car testing that out?"
- "No, I just read comp.lang.python."

Stefan

--
http://mail.python.org/mailman/listinfo/python-list

Re: Where is the man page of python library

[Grant Edwards]

On 2010-07-23, Steven D'Aprano  wrote:
> On Thu, 22 Jul 2010 20:18:43 -0700, march wrote:
>
>> Hi, guys.
>> 
>> As a regular user of python, I am often annoyed by the fact that the
>> official python docementation is too short and too simple to satisfy my
>> requirement.
>
> Python is a volunteer effort. If the docs don't suit your requirements, 
> we're grateful for patches.
>
>> While working with socket, I want to know every detail about every API. 
>> I can easilly achieve that by reading man page if the language is C. But
>> It seems that the Python story is different.
>
> Python is open source. Where the documentation is silent, the ultimate 
> authority is the source code. Particularly if the code is a thin wrapper 
> around the C library, which I expect (but don't know for sure) the socket 
> code will be.
>
>
>> For the interface recv(), all I got is only three sentences. "
>> Receive data from the socket. The return value is a string representing
>> the data received. The maximum amount of data to be received at once is
>> specified by bufsize. "
>> http://docs.python.org/library/socket.html#socket.socket.recv
>> 
>> What if the call fail? 
>
> You will get an exception, just like the page says:
>
> All errors raise exceptions. The normal exceptions for 
> invalid argument types and out-of-memory conditions can be 
> raised; errors related to socket or address semantics raise 
> the error socket.error.
>
>
>> What if the peer close the socket? 
>
> You will get an exception,

Nope.  You read a value of "".

> just like the Fine Manual says.

If it does say that, it needs to be fixed.

>> I hate this documentation!

Then quit bitching and submit a patch.

> Don't blame the documentation for your failure to read it. It's true
> that it could be improved, but most of your questions were answered
> by the page you linked to.

-- 
Grant Edwards   grant.b.edwardsYow! I am having FUN...
  at   I wonder if it's NET FUN or
  gmail.comGROSS FUN?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Where is the man page of python library

[Steven D'Aprano]

On Fri, 23 Jul 2010 14:26:27 +, Grant Edwards wrote:

>>> What if the peer close the socket?
>>
>> You will get an exception,
> 
> Nope.  You read a value of "".

Thank you for the correction.

 
>> just like the Fine Manual says.
> 
> If it does say that, it needs to be fixed.

No, I apparently just made it up.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: detect endianness of a binary with python

[Robert Kern]

On 7/23/10 12:44 AM, Tim Roberts wrote:

I wouldn't use os.system with grep and evaluate the return code.  Instead
I'd use subprocess.Popen("file") and read the text output of the
commdn directly.  By parsing that string, I can extract all kinds of
interesting information.


Small correction: subprocess.Popen(["file", our_image_filename])

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list

Re: Where is the man page of python library

[rantingrick]

On Jul 23, 9:49 am, Steven D'Aprano  wrote:
> On Fri, 23 Jul 2010 14:26:27 +, Grant Edwards wrote:
> > If it does say that, it needs to be fixed.
>
> No, I apparently just made it up.

Yes and i'll bet you've *never* just "made up" anything to scaffold
your arguments? . Since this trait is one of the major
personality flaws of the deeply religious... are you *sure* your *not*
a religious man Steven? We've also seen our fair share of haughty
arrogance and sanctimoniousness from time to time. Just observations
really. Sorry, i guess I just wanted to put a big neon sign around
your mistake. Of course now we both have our eyes poked out!

In the land of the blind, the one eyed man is king! ;-)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Where is the man page of python library

[James Mills]

On Sat, Jul 24, 2010 at 1:42 AM, rantingrick  wrote:
> In the land of the blind, the one eyed man is king! ;-)

RIck, your comments don't really help the situation. Really.

-- 
-- James Mills
--
-- "Problems are solved by method"
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Easy questions from a python beginner

[Stephen Hansen]

On 7/23/10 2:05 AM, Steven D'Aprano wrote:
> On Thu, 22 Jul 2010 21:23:05 -0700, Stephen Hansen wrote:
> 
>> On 7/22/10 7:47 PM, wheres pythonmonks wrote:
> [...]
>>> The truth is that I don't intend to use these approaches in anything
>>> serious.  However, I've been known to do some metaprogramming from time
>>> to time.
>>
>> Depending on how you define "metaprogramming", Python is pretty
>> notoriously ill-suited towards the task (more, its basically considered
>> a feature it doesn't let you go there) -- or, it allows you to do vast
>> amounts of stuff with its few dark magic hooks. I've never had a
>> satisfying definition of metaprogramming that more then 50% of any group
>> agree with, so I'm not sure which you're looking for. :)
> 
> I disagree strongly at your characterisation that Python is notorious for 
> being ill-suited towards metaprogramming. I'd say the complete opposite 
> -- what is considered dark and scary metaprogramming tasks in other 
> languages is considered too ordinary to even mention in Python.

I rather think you missed my point entirely.

'Depending on how you define "metaprogramming"'

The 'Depending' was the most important part of that sentence. You go on
to talk about runtime modification of classes and the like. That, Python
lets you do quite readily, and then go on to do crazy amounts of without
any significant effort.

That's one definition of metaprogramming. That one Python does well.

The other involves things like macros, or things where you basically
write a new sub-language in the language itself to achieve some commonly
desired task more efficiently (or just more succinctly). That's another
definition of metaprogramming: where its not so much structures
(classes, etc) which are modified at runtime (which Python lets you do
readily), but the syntax or semantics of the language itself. That
Python isn't game for.

> [...]
>> But! What it doesn't let you do is get clever with syntax and write a
>> sort of "simplified" or domain specific language to achieve certain
>> sorts of repetitive tasks quickly. You always end up writing normal
>> Python that looks like all other Python.
> 
> Exactly... 90% of the time that you think you want a DSL, Python beat you 
> to it.

Yet, that is a branch of what is considered metaprogramming that the OP
seems to be asking for, that we do not offer any sort of real support
for. I was making this distinction.

-- 

   Stephen Hansen
   ... Also: Ixokai
   ... Mail: me+list/python (AT) ixokai (DOT) io
   ... Blog: http://meh.ixokai.io/



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Unicode error

[Thomas Jollans]

On 07/23/2010 12:56 PM, dirknbr wrote:
> To give a bit of context. I am using twython which is a wrapper for
> the JSON API
> 
>  
> search=twitter.searchTwitter(s,rpp=100,page=str(it),result_type='recent',lang='en')
> for u in search[u'results']:
> ids.append(u[u'id'])
> texts.append(u[u'text'])
> 
> This is where texts comes from.
> 
> When I then want to write texts to a file I get the unicode error.

So your data is unicode? Good.

Well, files are just streams of bytes, so to write unicode data to one
you have to encode it. Since Python can't know which encoding you want
to use (utf-8, by the way, if you ask me), you have to do it manually.

something like:

outfile.write(text.encode('utf-8'))

-- 
http://mail.python.org/mailman/listinfo/python-list

time between now and the next 2:30 am?

[Jim]

How can I calculate how much time is between now and the next 2:30
am?  Naturally I want the system to worry about leap years, etc.

Thanks,
Jim
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: time between now and the next 2:30 am?

[Neil Cerutti]

On 2010-07-23, Jim  wrote:
> How can I calculate how much time is between now and the next
> 2:30 am?  Naturally I want the system to worry about leap
> years, etc.

You need the datetime module. Specifically, a datetime and
timedelta object.

-- 
Neil Cerutti
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Easy questions from a python beginner

[Thomas Jollans]

On 07/23/2010 12:34 AM, wheres pythonmonks wrote:
> 2.  Is there a better way to loopup by id?  I'm not very familiar with
> sys.exc_info, but creating the id->name hash each time seems like
> overkill.

I just had the most horrendous idea. Really, looking up objects by ID,
or even swapping two objects, isn't that difficult if you do some C
black magic. Don't try this in front of the kids.

I wrote a little module, called "hell":

Python 3.1.2 (release31-maint, Jul  8 2010, 09:18:08)
[GCC 4.4.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> from hell import swap, getptr
>>> dir

>>> len

>>> swap(dir, len)
>>> dir

>>> len

>>> a = "this was a"
>>> b = "this was b, hell yeah"
>>> (a, b, id(a), id(b))
('this was a', 'this was b, hell yeah', 32417752, 32418144)
>>> tpl = (a, b, id(a), id(b))
>>> tpl
('this was a', 'this was b, hell yeah', 32417752, 32418144)
>>> swap(a, b)
>>> tpl
('this was b, hell yeah', 'this was a', 32417752, 32418144)
>>> getptr(32417752)
'this was b, hell yeah'
>>>


The code is below. Use it under the terms of the WTFPL version 2.
(http://sam.zoy.org/wtfpl/)

 -- Thomas

PS: all of this is a very BAD IDEA. But interesting, in a way.


# setup.py 
from distutils.core import setup, Extension

hellmodule = Extension('hell',
   sources = ['hellmodule.c'])

setup (name = 'hellmodule',
   version = '0.1-apocalypse',
   description = 'Functions from hell. Never ever use.',
   ext_modules = [hellmodule])



# hellmodule.c ##
#define PY_SSIZE_T_CLEAN
#include 

static PyObject *swap(PyObject *self, PyObject *args);
static PyObject *getptr(PyObject *self, PyObject *args);

static PyMethodDef hell_methods[] = {
{"swap",  &swap,  METH_VARARGS, ""},
{"getptr", &getptr, METH_VARARGS, ""},
{NULL, NULL, 0, NULL}
};

static struct PyModuleDef hell_module = {
PyModuleDef_HEAD_INIT,
"hell",/* module name */
"functions from hell. never use.", /* doc */
-1,
hell_methods
};

PyMODINIT_FUNC
PyInit_hell(void)
{
return PyModule_Create(&hell_module);
}

static PyObject *
swap(PyObject *self, PyObject *args)
{
PyObject *obj1, *obj2;
Py_ssize_t len;

PyObject *temp;


if (!PyArg_ParseTuple(args, "OO", &obj1, &obj2)) {
return NULL;
}

len = obj1->ob_type->tp_basicsize;
if (obj2->ob_type->tp_basicsize != len) {
PyErr_SetString(PyExc_TypeError,
"types have different sizes (incompatible)");
return NULL;
}

temp = PyMem_Malloc(len);
memcpy(temp, obj1, len);
memcpy(obj1, obj2, len);
memcpy(obj2, temp, len);
obj2->ob_refcnt = obj1->ob_refcnt;
obj1->ob_refcnt = temp->ob_refcnt;

Py_INCREF(Py_None);
return Py_None;
}

static PyObject *
getptr(PyObject *self, PyObject *args)
{
unsigned long lId;
PyObject *retv;

if (!PyArg_ParseTuple(args, "k", &lId)) {
return NULL;
}

retv = (PyObject*) lId;
Py_INCREF(retv);
return retv;
}


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Easy questions from a python beginner

[Thomas Jollans]

On 07/23/2010 07:13 PM, Thomas Jollans wrote:
> On 07/23/2010 12:34 AM, wheres pythonmonks wrote:
>> 2.  Is there a better way to loopup by id?  I'm not very familiar with
>> sys.exc_info, but creating the id->name hash each time seems like
>> overkill.
> 
> I just had the most horrendous idea. Really, looking up objects by ID,
> or even swapping two objects, isn't that difficult if you do some C
> black magic. Don't try this in front of the kids.
> 
> I wrote a little module, called "hell":
> 
> Python 3.1.2 (release31-maint, Jul  8 2010, 09:18:08)
> [GCC 4.4.4] on linux2
> Type "help", "copyright", "credits" or "license" for more information.

 from hell import swap, getptr
 dir
> 
 len
> 
 swap(dir, len)
 dir
> 
 len
> 
 a = "this was a"
 b = "this was b, hell yeah"
 (a, b, id(a), id(b))
> ('this was a', 'this was b, hell yeah', 32417752, 32418144)
 tpl = (a, b, id(a), id(b))
 tpl
> ('this was a', 'this was b, hell yeah', 32417752, 32418144)
 swap(a, b)
 tpl
> ('this was b, hell yeah', 'this was a', 32417752, 32418144)
 getptr(32417752)
> 'this was b, hell yeah'



The great thing about this is that it can illustrate, in a very perverse
manner, some lovely facets of Python. For example: combined hash and
equality checks when accessing sets and dicts:

Python 3.1.2 (release31-maint, Jul  8 2010, 09:18:08)
[GCC 4.4.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from hell import swap
>>> s1 = 'string1'
>>> s2 = 'string2'
>>> s = {s1, s2}
>>> swap(s1, s2)
>>> s
{'string1', 'string2'}
>>> s1 in s
False
>>> s2 in s
False
>>> s1 in list(s)
True
>>> s2 in list(s)
True
>>>

-- 
http://mail.python.org/mailman/listinfo/python-list

Function closure inconsistency

[SeanMon]

I was playing around with Python functions returning functions and the
scope rules for variables, and encountered this weird behavior that I
can't figure out.

Why does f1() leave x unbound, but f2() does not?

def f1():
x = 0
def g():
x += 1
return x
return g1

def f2():
x = []
def g():
x.append(0)
return x
return g

a = f1()
b = f2()

a() #UnboundLocalError: local variable 'x' referenced before
assignment
b() #No error, [0] returned
b() #No error, [0, 0] returned
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Function closure inconsistency

[Benjamin Kaplan]

On Fri, Jul 23, 2010 at 11:30 AM, SeanMon  wrote:
>
> I was playing around with Python functions returning functions and the
> scope rules for variables, and encountered this weird behavior that I
> can't figure out.
>
> Why does f1() leave x unbound, but f2() does not?
>
> def f1():
>    x = 0
>    def g():
>        x += 1
>        return x
>    return g1
>
> def f2():
>    x = []
>    def g():
>        x.append(0)
>        return x
>    return g
>
> a = f1()
> b = f2()
>
> a() #UnboundLocalError: local variable 'x' referenced before
> assignment
> b() #No error, [0] returned
> b() #No error, [0, 0] returned
> --



It's not closure related at all. Same thing happens at the module level.

x = 0
def f1() :
   x += 1
#gives UnboundLocalError

x = []
def f2() :
   x.append(1)
#succeeds.

The reason for it is that if you have any assignments to the variable
in the function, Python creates a new local variable for it. x += 1 is
an assignment, not a modification. Python 2.x allows you to assign to
the global scope (using the global keyword) but support for assigning
to the outer function's scope wasn't added until Python 3 (with the
nonlocal keyword)


def f1():
   x = 0
   def g():
   nonlocal x
   x += 1
   return x
   return g1

>
> http://mail.python.org/mailman/listinfo/python-list
-- 
http://mail.python.org/mailman/listinfo/python-list

A portable LISP interpreter that includes all the major list-processing functions is described. A complete, annotated listing of the program's code, written in PASCAL, is included.

[Emmy Noether]

Title   Portable LISP interpreter
Creator/Author  Cox, L.A. Jr. ; Taylor, W.P.
Publication Date1978 May 31
OSTI Identifier OSTI ID: 7017786
Report Number(s)UCRL-52417
DOE Contract Number W-7405-ENG-48
Resource Type   Technical Report
Research OrgCalifornia Univ., Livermore (USA). Lawrence Livermore
Lab.
Subject 99 GENERAL AND MISCELLANEOUS//MATHEMATICS, COMPUTING, AND
INFORMATION SCIENCE; COMPUTER CODES; L CODES; COMPUTERS; PROGRAMMING;
PROGRAMMING LANGUAGES
Description/AbstractA portable LISP interpreter that includes all the
major list-processing functions is described. A complete, annotated
listing of the program's code, written in PASCAL, is included.
Country of Publication  United States
LanguageEnglish
Format  Medium: X; Size: Pages: 21
AvailabilityDep. NTIS, PC A02/MF A01.
System Entry Date   2008 Feb 12
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: time between now and the next 2:30 am?

[Jim]

Thanks; I'll have a look, and a think.

Jim
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Function closure inconsistency

[Dave Angel]

SeanMon wrote:

I was playing around with Python functions returning functions and the
scope rules for variables, and encountered this weird behavior that I
can't figure out.

Why does f1() leave x unbound, but f2() does not?

def f1():
x = 0
def g():
x += 1
return x
return g1

def f2():
x = []
def g():
x.append(0)
return x
return g

a = f1()
b = f2()

a() #UnboundLocalError: local variable 'x' referenced before
assignment
b() #No error, [0] returned
b() #No error, [0, 0] returned

  
Your example is more complex than needed.  The symptom doesn't need a 
function closure.


>>> def g():
...x += 1
...return x
...
>>> g()
Traceback (most recent call last):
 File "", line 1, in 
 File "", line 2, in g
UnboundLocalError: local variable 'x' referenced before assignment
>>> def f():
...x.append(0)
...return x
...
>>> x = [3,5]
>>> f()
[3, 5, 0]
>>>


The difference between the functions is that in the first case, x is 
reassigned; therefore it's a local.  But it's not defined before that 
line, so you get the ref before assign error.


In the second case, append() is an in-place operation, and doesn't 
create a local variable.


DaveA

--
http://mail.python.org/mailman/listinfo/python-list

Light-weight/very-simple version control under Windows using Python?

[python]

I have some very simple use cases[1] for adding some version control
capabilities to a product I'm working on. My product uses simple, text
(UTF-8) based scripts that are independent of one another. I would like
to "version control" these scripts on behalf of my users. By version
control, I mean *very-simple* version control with no branching or
merging - just the ability to store, list and restore a specific version
of a file. The data store should be a local file with the ability to
upsize to a multi-user database in the future.

I'm looking for recommendations on possible solutions:

1. Use an existing version control utility. There are lots of options
here(!), any recommendations on a light weight, open source one that
xcopy installs under Windows with lots of command line options?

2. Interface to a hosted version control system (SaaS) that provides a
RESTful API. Any recommendations here?

3. Build this capability myself using Python and Python's DBI layer to
store files in a local SQLite database at first (but with the ability to
upsize to a real client server database in the future). Seems like a fun
project to work on, but also smells like I'd be re-inventing the wheel
with very little value added other than simpler deployment?

Any suggestions appreciated.

Malcolm

[1] Here's my use cases:

1. Check a file in with optional comment and username; ideally
get a version number that can be used to reference this specific
check-in in the future.

2. Get a history listing of all checkins for a specific file
(version number, timestamp, file size, user, comment)

3. Check out a specific version of a file by version number.

4. Delete checked-in versions by version number, date range,
and/or username.

5. (Optional) Diff 2 versions of a file by version number and
return diff in richly formatted format that visually shows
changes via color and font effects (strikethru) (I'm thinking
of using BeyondCompare for this if not present in a simple
version control tool)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Light-weight/very-simple version control under Windows using Python?

[Thomas Jollans]

On 07/23/2010 10:35 PM, pyt...@bdurham.com wrote:
> 1. Use an existing version control utility. There are lots of options
> here(!), any recommendations on a light weight, open source one that
> xcopy installs under Windows with lots of command line options?

You could just go with Mercurial, you know. Very popular, powerful,
portable, distributed (so you need no server infrastructure...), free
software of course, and written in Python, so interfacing it from Python
isn't that hard. It also scales up and down very well, so you can easily
change your mind about what you want it to do later if you want to.

> 1. Check a file in with optional comment and username; ideally
> get a version number that can be used to reference this specific
> check-in in the future.

trivial.

> 2. Get a history listing of all checkins for a specific file
> (version number, timestamp, file size, user, comment)

trivial as in that's the whole point.

> 3. Check out a specific version of a file by version number.

Mercurial, and the other distributed VCS, don't technically track
changes to files, they track changes to the whole repository (think
"directory" if you're not acquainted with DVCS) -- as with any other
existing solution you might use, you'll have to think a bit more on its
(whatever tool you end up using) terms.

> 4. Delete checked-in versions by version number, date range,
> and/or username.

Do you really want to do this? It's possible, even with Mercurial, but
do you really want to create history just to delete it later? Think
about it.

> 5. (Optional) Diff 2 versions of a file by version number and
> return diff in richly formatted format that visually shows
> changes via color and font effects (strikethru) (I'm thinking
> of using BeyondCompare for this if not present in a simple
> version control tool)

Yeah, just look for a nice graphical diff tool. KDiff3 springs to mind,
I think that's what comes bundled with TortoiseHg on Windows.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: C interpreter in Lisp/scheme/python

[George Neuner]

On Fri, 23 Jul 2010 15:10:16 +0200, francogrex 
wrote:

>Unfortunately many so-called experts in the field look down 
>on newbies and mistreat them (in any programming language forum),
>forgetting in the process that they were also at a certain time
>newbies until some gentle and nice enough teachers took the 
>trouble to educate them. 

I don't think it's accurate to say that [some] experts really "scorn"
newbies, but I do agree that newbies are occasionally mistreated.  

One thing newbies have to realize is that on Usenet you are quite
likely to be talking to people who were there at the beginning and, of
necessity, are largely self educated in whatever the subject matter
might be.  Many - I'd even say most - are happy to clarify
understanding and help with complicated problems, but there is a
general expectation that newbies have some basic research skills and
that they have tried to solve their problem before asking for help.

Unfortunately, there is a small percentage of people who think Usenet
and other online forums are for answering homework questions or for
digging out of a jam at work.  Getting help depends a lot on how the
question is asked: strident pleas for quick help or demands for an
answer are immediate red flags, but so are questions that begin with
"X is crap, why can't I do ..." and even seemingly polite questions
that are vague or unfocused (possibly indicating little or no thought
behind them) or posts which are directed to a large number of groups
(such as this thread we're in now).  

And, of course, in the language forums, drawing comparisons to
non-subject languages is generally considered rude except when done to
illustrate a relevant discussion point.  Introducing irrelevant
comparisons, deliberately proselytizing X in a Y group or doing a lot
of complaining about the subject language is bound to attract disdain.

As the Internet has grown, the absolute number of people in that
"small percentage" has grown as well.  A newbie can simply be unlucky
enough to ask a question at the wrong time.  If there has been a
recent rash of problem posts then experts may accidentally respond
negatively to a legitimate question.

Of course, there are cross-cultural issues too.  Many of the technical
groups are English-language.  English, even when polite, can seem
harsh and/or abrupt to non-native speakers.

On the whole, moderated groups are more conducive to helping newbies
because the moderator(s) filter obvious red flag posts.

And, finally, newbies themselves should realize that experts are
donating time to answer questions and do get frustrated answering the
same questions over and over.  They should not be offended by "cold"
responses that direct them to FAQs or that just give links to study
material.  Newbies who need hand-holding or warm welcoming responses
filled with detail should go find a tutor.


> ... you have the bad "professors" who are freaks 
>(probably they have a lot of problems at home, their wives 
>screwing all the males on the block, daughters drug addicts etc) 
>and want to take their hatred out on you,

Unquestionably, there are experts who need their dosages adjusted. But
the same can be said for some percentage of other users too.

OTOH, newbies often aren't in the position to know who is an expert
... obviously, anyone able to correctly answer their question knows
more about that specific issue.  That doesn't necessarily qualify the
responder as an "expert".  Some people get defensive at the edges of
their comfort zones.


Just some thoughts. YMMV.
George
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: ANN: blist 1.2.0

[Daniel Stutzbach]

On Wed, Jul 21, 2010 at 9:47 AM, Daniel Stutzbach <
dan...@stutzbachenterprises.com> wrote:

> What's new?
> ---
>
> - blist.sort() is now *substantially* faster than list.sort() when using
> int or float keys (O(n) vs. O(n log n))
> - The sortedset, sortedlist, and sorteddict types have been revamped for
> better compatibility with the standard library's types.
> - Comprehensive reference documentation is now available at
> http://stutzbachenterprises.com/blist/
> - Numerous other speed improvements
>

Quick addendum:

I just uploaded blist 1.2.1 which fixes a compilation error on BSD-derived
systems (notably MacOS X).
--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: non-blocking IO EAGAIN on write

[Nobody]

On Fri, 23 Jul 2010 10:45:32 +0200, Thomas Guettler wrote:

> I use non-blocking io to check for timeouts. Sometimes I get EAGAIN
> (Resource temporarily unavailable) on write(). My working code looks
> like this. But I am unsure how many bytes have been written to the pipe
> if I get an EAGAIN IOError.

It should be zero; that's how the underlying system calls behave. If any
bytes are read/written, the number of bytes are returned. The call
only blocks (or fails with EAGAIN if non-blocking I/O is enabled) if it
isn't possible to read/write any data.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Unicode error

[Nobody]

On Fri, 23 Jul 2010 10:42:26 +, Steven D'Aprano wrote:

> Don't write bare excepts, always catch the error you want and nothing 
> else.

That advice would make more sense if it was possible to know which
exceptions could be raised. In practice, that isn't possible, as the
documentation seldom provides this information. Even for the built-in
classes, the documentation is weak in this regard; for less important
modules and third-party libraries, it's entirely absent.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Unicode error

[Benjamin Kaplan]

On Fri, Jul 23, 2010 at 2:46 PM, Nobody  wrote:
> On Fri, 23 Jul 2010 10:42:26 +, Steven D'Aprano wrote:
>
>> Don't write bare excepts, always catch the error you want and nothing
>> else.
>
> That advice would make more sense if it was possible to know which
> exceptions could be raised. In practice, that isn't possible, as the
> documentation seldom provides this information. Even for the built-in
> classes, the documentation is weak in this regard; for less important
> modules and third-party libraries, it's entirely absent.
>

You still don't want to use bare excepts.People tend to get rather
annoyed when you handle KeyboardInterrupts and SystemExits like you
would a UnicodeError. Use Exception if you don't know what exceptions
can be raised.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Unicode error

[Thomas Jollans]

On 07/23/2010 11:46 PM, Nobody wrote:
> On Fri, 23 Jul 2010 10:42:26 +, Steven D'Aprano wrote:
> 
>> Don't write bare excepts, always catch the error you want and nothing 
>> else.
> 
> That advice would make more sense if it was possible to know which
> exceptions could be raised. In practice, that isn't possible, as the
> documentation seldom provides this information. Even for the built-in
> classes, the documentation is weak in this regard; for less important
> modules and third-party libraries, it's entirely absent.
> 

In practice, at least in Python, it tends to be better to work the
"other way around": first, write code without exception handlers. Test.
If you get an exception, there are really two possible reactions:


 1. "WHAT??"
  => This shouldn't be happening. Rather than catching everything,
 fix your code, or think it through until you reach conclusion
 #2 below.

2. "Ah, yes. Of course. I should check for that."
  => No problem! You're staring at a traceback right now, so you
 know the exception raised.

If you know there should be an exception, but you don't know which one,
it should be trivial to create condition in which the exception arises,
should it not? Then, you can handle it properly, without resorting to
guesswork or over-generalisations.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Easy questions from a python beginner

[Terry Reedy]

On 7/23/2010 4:10 AM, Steven D'Aprano wrote:


Using exec or friends to avoid the overhead of function calls is like
pushing your car to work to avoid the overhead of having to get in and
out of the car.


And, of course, exec *is* a function call (explicitly in 3.x, but 
somewhere also in the innards of 2.x).


Thanks for the laugh of the day.

--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Jesus in the Glorious Qur'an ------ The True Message of Jesus Christ

[nais-saudi]

Jesus in the Glorious Qur'an -- The True Message of Jesus Christ

The Qur’an tells us a lot of wonderful things about Jesus. As a
result, believers in the Qur’an love Jesus, honour him, and believe in
him. In fact, no Muslim can be a Muslim unless he or she believes in
Jesus, on whom be peace.

The Qur’an says that Jesus was born of a virgin, that he spoke while
he was still only a baby, that he healed the blind and the leper by
God’s leave, and that he raised the dead by God’s leave.

What then is the significance of these miracles? First, the virgin
birth. God demonstrates his power to create in every way. God created
everyone we know from a man and a woman. But how about Adam, on whom
be peace? God created him from neither a man nor a woman. And Eve from
only a man, but not a woman. And, finally, to complete the picture,
God created Jesus from a woman, but not a man.

What about the other miracles? These were to show that Jesus was not
acting on his own behalf, but that he was backed by God. The Qur’an
specifies that these miracles were performed by God’s leave. This may
be compared to the Book of Acts in the Bible, chapter 2, verse 22,
where it says that the miracles were done by God to show that he
approved of Jesus. Also, note that Jesus himself is recorded in the
Gospel of John to have said, “I can do nothing of my own
authority” (5:30). The miracles, therefore, were done not by his own
authority, but by God’s authority.

What did Jesus teach? The Qur’an tells us that Jesus came to teach the
same basic message which was taught by previous prophets from God—that
we must shun every false god and worship only the one true God. Jesus
taught that he is the servant and messenger of that one true God, the
God of Abraham. These Quranic teachings can be compared with the Bible
( Mark 10:18; Matthew 26:39; John 14:28, 17:3, and 20:17) where Jesus
teaches that the one he worshipped is the only true God. See also
Matthew 12:18; Acts 3:13, and 4:27 where we find that his disciples
knew him as Servant of God.

The Qur’an tells us that some of the Israelites rejected Jesus, and
conspired to kill him, but Allah (God) rescued Jesus and raised him to
Himself. Allah will cause Jesus to descend again, at which time Jesus
will confirm his true teachings and everyone will believe in him as he
is and as the Qur’an teaches about him.

Jesus is the Messiah. He is a word from Allah, and a spirit from Him.
He is honoured in this world and in the hereafter, and he is one of
those brought nearest to Allah.

Jesus was a man who spoke the truth which he heard from God. This can
be compared with the Gospel According to John where Jesus says to the
Israelites: “You are determined to kill me, a man who has told you the
truth that I heard from God” (John 8:40).


The Virgin Birth of Jesus
Muslims believe in the virgin birth of Jesus. When the angels
announced to Mary (peace be upon her) about Allah’s promise that she
will have a son, she was surprised, since she was a virgin. “How can
this be?” she thought. She was reminded that it is easy for Allah to
create whatever he wills.

She said: My Lord! How can I have a child when no mortal hath touched
me? He said: So (it will be). Allah createth what He will. If He
decreeth a thing, He saith unto it only: Be! and it is (Qur’an 3:47).

It is not difficult for Allah to do anything he wants. He can create a
child with both human parents or only one. No miracle is beyond His
power. After all, He had created Adam (peace be upon him) from neither
a man nor a woman. He created the rest of us from both man and woman.
What is so hard if Allah decides to create a human being from a woman
only? He only commands “Be!” and it occurs.

Some people think that since Jesus, peace be upon him, had no human
father then God must be his father. The Qur’an rejects this view. The
position of Jesus with Allah is comparable to the position of Adam
with Allah. Just because Adam had no human parent does not mean we
should call him the Son of God.

Lo! the likeness of Jesus with Allah is as the likeness of Adam. He
created him from dust, then He said unto him: Be! and he is. (Qur’an
3:59).

According to the Qur’an, everyone except Allah are His servants.

And they say: the Beneficent hath taken unto Himself a Son. Assuredly
ye utter a disastrous thing, whereby almost the heavens are torn, and
the earth is split asunder and the mountains fall to ruins, that ye
ascribe to the Beneficent a son, when it is not meet for (the Majesty
of) the Beneficent that He should chose a son. There is none in the
heavens and the earth but cometh unto the Beneficent as a slave.
(Qur’an 19:88-93)


The Miracles of Jesus
According to the Qur’an, Jesus, on whom be peace, performed the
following miracles by Allah’s leave:

1. Spoke while he was only a baby.
2. Healed those born blind.
3. Healed the lepers.
4. Revived the dead.
5. Breathed life into a bird made of clay.

In the Qur’an Allah quotes Jesus, peace be upon him, as saying:

Re: Function closure inconsistency

[Terry Reedy]

On 7/23/2010 2:30 PM, SeanMon wrote:

I was playing around with Python functions returning functions and the
scope rules for variables, and encountered this weird behavior that I
can't figure out.

Why does f1() leave x unbound, but f2() does not?

def f1():
 x = 0
 def g():

In 3.x, add
   nonlocal x

 x += 1
 return x
 return g1

You meant g

def f1():
 x = 0
 def g():
  nonlocal x
  x += 1
  return x
 return g
f=f1()
print(f())
print(f())
print(f())
print(f())

1
2
3
4
--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Unicode error

[Terry Reedy]

On 7/23/2010 5:46 PM, Nobody wrote:

On Fri, 23 Jul 2010 10:42:26 +, Steven D'Aprano wrote:


Don't write bare excepts, always catch the error you want and nothing
else.


That advice would make more sense if it was possible to know which
exceptions could be raised. In practice, that isn't possible, as the
documentation seldom provides this information. Even for the built-in
classes, the documentation is weak in this regard; for less important
modules and third-party libraries, it's entirely absent.


I intend to bring that issue up on pydev list sometime. But in the 
meanwhile, once you get an error, you know what it is. You can 
intentionally feed code bad data and see what you get. And then maybe 
add a test to make sure your code traps such errors.


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: time between now and the next 2:30 am?

[David Bolen]

Neil Cerutti  writes:

> On 2010-07-23, Jim  wrote:
>> How can I calculate how much time is between now and the next
>> 2:30 am?  Naturally I want the system to worry about leap
>> years, etc.
>
> You need the datetime module. Specifically, a datetime and
> timedelta object.

Although it sounds like the question is to derive the timedelta value,
so it's not known up front.  That's a little trickier since you'd need
to construct the datetime object for "next 2:30 am" to subtract "now"
from to get the delta.  But that requires knowing when the next day
is, thus dealing with month endings.  Could probably use the built-in
calendar module to help with that though.

For the OP, you might also take a peek at the dateutil third party
module, and its relativedelta support, which can simplify the creation
of the "next 2:30 am" datetime object.

Your case could be handled by something like:

from datetime import datetime
from dateutil.relativedelta import relativedelta

target = datetime.now() + relativedelta(days=+1, hour=2, minute=30,
second=0, microsecond=0)
remaining = target - datetime.now()

This ends up with target being a datetime instance for the next day at
2:30am, and remaining being a timedelta object representing the time
remaining, at least as of the moment of its calculation.

Note that relativedelta leaves fields alone that aren't specified, so
since datetime.now() includes down to microseconds, I clear those
explicitly).  Since you really only need the date, you could also use
datetime.date.today() instead as the basis of the calculation and then
not need second/microsecond parameters to relativedelta.

-- David
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: time between now and the next 2:30 am?

[Christian Heimes]

> Your case could be handled by something like:
> 
> from datetime import datetime
> from dateutil.relativedelta import relativedelta
> 
> target = datetime.now() + relativedelta(days=+1, hour=2, minute=30,
> second=0, microsecond=0)
> remaining = target - datetime.now()

You don't need the dateutil package for the trick:

>>> dt = datetime(2010, 1, 1, 1, 0)
>>> str(dt)
'2010-01-01 01:00:00'
>>> next = dt.replace(hour=2, minute=30)
>>> next - dt
datetime.timedelta(0, 5400)
>>> (next - dt).seconds
5400

>>> dt = datetime(2010, 1, 1, 3, 0)
>>> next = dt.replace(hour=2, minute=30)
>>> next - dt
datetime.timedelta(-1, 84600)
>>> (next - dt).seconds
84600

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: time between now and the next 2:30 am?

[Jim]

Thank you again to everyone; I greatly appreciate the help.  I ended
with essentially what Christian advised and it seems to work fine.
FWIW, below is the rouine (I couldn't figure out how to do it without
the kludgy x variable).  The background is that this is a Django
context processor that makes available the time until the next 2:30
because that is when the web site is rebuilt.

I hope the code isn't too much mangled by the online editor,
Jim

.

import time, datetime
from django.conf import settings

WARNING_MINS=10  # How many mins before update time should start
warning?
WARNING_SECS=60*WARNING_MINS

def current_time_processor(req):
"""Add current time information to the Context
"""
# Calculate time until the next 2:30 am
x=datetime.datetime(2010,7,23)  # actual date doesn't matter?
dt_localtime=x.fromtimestamp(time.time())
 
dt_twothirty=dt_localtime.replace(hour=settings.UPDATE_TIME_HOURS,minute=settings.UPDATE_TIME_MINS,second=0,microsecond=0)
dd_diff=dt_twothirty-dt_localtime
secs_until_twothirty=dd_diff.seconds
if secs_until_twothirtyhttp://mail.python.org/mailman/listinfo/python-list

Socket performance

[Navkirat Singh]

Hey Everyone,

I had a question, programming sockets, what are the things that would  
degrade performance and what steps could help in a performance boost?  
I would also appreciate being pointed to some formal documentation or  
article.


I am new to this.

Warm regards,
Nav
--
http://mail.python.org/mailman/listinfo/python-list

Re: loading configuration files that are themselves python

[Lawrence D'Oliveiro]

In message <7j8w5tylmw@rapun.sel.cam.ac.uk>, Matthew Vernon wrote:

> Is there a more idiomatic way of loading in a configuration file
> that's python code ...

Is it really a good idea to have a configuration language that’s Turing-
complete?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python as a scripting language. Alternative to bash script?

[Lawrence D'Oliveiro]

In message , Tim Harig wrote:

> On 2010-07-05, Lawrence D'Oliveiro 
> wrote:
>>
>> I’ve never come across any system where you could string together
>> multiple GUI apps, or even multiple GUI operations in the same app, in
>> any sensible or effective way at all. GUIs just aren???t designed to work
>> that way.
> 
> You can if they are designed to be used externally.  COM is an execellent
> example of this ...

But COM is an IPC architecture, which has nothing to do with GUI apps. 
Conflating the two leads to precisely the sort of inflexibility that is 
characteristic of GUIs. For example, things happening with the GUI on-screen 
while a script is running that are at best distracting to the user, or at 
worst vulnerable to breaking if the user accidentally presses a key or 
clicks in the wrong place.

> I have worked with complex business process automation centered around
> Excel and I have automated some really ugly AJAX style web based
> applications that would only work inside of IE6 by accessing IE through
> its COM interface.

If those apps really were using AJAX, then why did you need to script a Web 
browser at all? You could just make direct HTTP requests to retrieve the 
data.

> Automating GUI applications requires interal access to the program through
> some kind of interface ...

Some kind of interface that bypasses the GUI. This is why so many Open 
Source apps are structured as a GUI front end to a completely separate 
command-line tool (e.g. MPlayer). That way a script can use the latter 
without having to go through the former.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Download Microsoft C/C++ compiler for use with Python 2.6/2.7 ASAP

[Lawrence D'Oliveiro]

In message , Robert Kern 
wrote:

> There are also utilities for mounting ISOs directly without burning them
> to a physical disk.

You need special utilities to do this??
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python as a scripting language. Alternative to bash script?

[Lawrence D'Oliveiro]

In message , Michael 
Torrie wrote:

> While it's possible to set up pipes and spawn programs in parallel to
> operate on the pipes, in practice it's simpler to tell subprocess.Popen
> to use a shell and then just rely on Bash's very nice syntax for setting
> up the pipeline.

Just be careful about properly escaping any special characters in the file 
names. See my other thread about escaping data, where some respondents have 
expressed a strong allergy to any form of embedding one language inside 
another.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Download Microsoft C/C++ compiler for use with Python 2.6/2.7 ASAP

[Robert Kern]

On 7/23/10 7:08 PM, Lawrence D'Oliveiro wrote:

In message, Robert Kern
wrote:


There are also utilities for mounting ISOs directly without burning them
to a physical disk.


You need special utilities to do this??


On at least some versions of Windows, Yes.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list

Re: time between now and the next 2:30 am?

[MRAB]

Jim wrote:

Thank you again to everyone; I greatly appreciate the help.  I ended
with essentially what Christian advised and it seems to work fine.
FWIW, below is the rouine (I couldn't figure out how to do it without
the kludgy x variable).  The background is that this is a Django
context processor that makes available the time until the next 2:30
because that is when the web site is rebuilt.

I hope the code isn't too much mangled by the online editor,
Jim

.

import time, datetime
from django.conf import settings

WARNING_MINS=10  # How many mins before update time should start
warning?
WARNING_SECS=60*WARNING_MINS

def current_time_processor(req):
"""Add current time information to the Context
"""
# Calculate time until the next 2:30 am
x=datetime.datetime(2010,7,23)  # actual date doesn't matter?
dt_localtime=x.fromtimestamp(time.time())
 

Why not:

dt_localtime = datetime.datetime.now()


dt_twothirty=dt_localtime.replace(hour=settings.UPDATE_TIME_HOURS,minute=settings.UPDATE_TIME_MINS,second=0,microsecond=0)


You're changing the time of day, but not the date. You might want to add
a day to the shutdown time if it's earlier than the current time.


dd_diff=dt_twothirty-dt_localtime
secs_until_twothirty=dd_diff.seconds
if secs_until_twothirty

--
http://mail.python.org/mailman/listinfo/python-list

Re: loading configuration files that are themselves python

[Ben Finney]

Lawrence D'Oliveiro  writes:

> In message <7j8w5tylmw@rapun.sel.cam.ac.uk>, Matthew Vernon wrote:
>
> > Is there a more idiomatic way of loading in a configuration file
> > that's python code ...
>
> Is it really a good idea to have a configuration language that’s Turing-
> complete?

I think not. Configuration files should be read as data; they should be
declarative only, not executable languages. That way, a different
program can read and parse them without having to be a parser for an
entire programming language.

As illustration of this point, I present the chronic headaches of the
Python ‘setup.py’ file. It tries to be both setup program *and*
configuration file, with the consequence that in the general case it's
impossible to get configuration information without executing the setup
program.

Recent work (lots of it!) by the Distutils team has gone into separating
the concerns so that the configuration data is all in a non-executable
data file; that work is ongoing, and it's been a hard lesson to learn.

-- 
 \   “If [a technology company] has confidence in their future |
  `\  ability to innovate, the importance they place on protecting |
_o__) their past innovations really should decline.” —Gary Barnett |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Socket performance

[MRAB]

Navkirat Singh wrote:

Hey Everyone,

I had a question, programming sockets, what are the things that would 
degrade performance and what steps could help in a performance boost? I 
would also appreciate being pointed to some formal documentation or 
article.


I am new to this.


Interleaving processing and sending/receiving might reduce throughput
because when you're processing the socket is idle (depending on how much
buffering there is). You could do the socket stuff in another thread so
that it's not waiting for the processing to finish while there is data
available, and use a queue to transfer the data between that thread and
the processing thread.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Socket performance

[Navkirat Singh]

Thanks for the info : ). I will look into it ! Right now I am having a  
strange problem. I am trying to use cookies and the import function  
returns an error:


I am using python 3:

from http import cookies

importError: No module named http

Is it my configuration or has something changed since the  
documentation was written? Sorry I might be asking too many question,  
I am pretty new to this stuff and kinda feel lost here and there : (


Thanks,
Nav



On 24-Jul-2010, at 6:34 AM, MRAB wrote:


Navkirat Singh wrote:

Hey Everyone,
I had a question, programming sockets, what are the things that  
would degrade performance and what steps could help in a  
performance boost? I would also appreciate being pointed to some  
formal documentation or article.

I am new to this.

Interleaving processing and sending/receiving might reduce throughput
because when you're processing the socket is idle (depending on how  
much
buffering there is). You could do the socket stuff in another thread  
so

that it's not waiting for the processing to finish while there is data
available, and use a queue to transfer the data between that thread  
and

the processing thread.
--
http://mail.python.org/mailman/listinfo/python-list


-- 
http://mail.python.org/mailman/listinfo/python-list

understanding the mro (long)

[Rolando Espinoza La Fuente]

TL;DR: if you want to stay sane, don't inherit two classes that share
same inheritance graph

I recently got puzzled by a bug from a legacy lib (ClientForm)
which have this code:

class ParseError(sgmllib.SGMLParseError,
 HTMLParser.HTMLParseError,
 ):
pass

And fails because takes __init__ from sgmllib and __str__ from HTMLParser
where __str__ uses attributes set by HTMLParser's init.

At first look, I thought was just matter to swap the inherit classes.
But a deeper
look take me to the python's mro reading:
http://www.python.org/download/releases/2.3/mro/

And to reproduce the error I code this:

class Foo(object):
def __init__(self, msg):
self.msg = msg

def __str__(self):
return 'Foo: ' + self.msg

class Bar(Exception):
def __init__(self, msg):
self.msg = msg

def __str__(self):
return 'Bar: ' + self.msg

class A(Exception):
pass

class B(RuntimeError):
pass

class AFoo(A, Foo): pass
class ABar(A, Bar): pass

class BFoo(B, Foo): pass
class BBar(B, Bar): pass

print AFoo('ok') # ok
print ABar('ok') # Bar: ok

print BFoo('ok') # ok
print BBar('fail') # AttributeError: ... not attribute 'msg'

# EOF

After running the code I was still confused. So I read carefully again
the mro stuff. And ended doing this inheritance tree:

   object (__init__, __str__)
  |\
  |Foo (__init__, __str__)
  |
 BaseException (__init__, __str__)
  |
  |
  |
  Exception (__init__)
 /| \
A| Bar (__init__, __str__)
  |
  StandardError (__init__)
  |
  |
  |
RuntimeError (__init__)
/
   B

Then I figure out the method resolution following the inheritance graph:
  * AFoo(A, Foo):
__init__ from Exception
__str__  from BaseException

  * ABar(A, Bar):
__init__ from Bar
__str__  from Bar

  * BFoo(B, Foo):
__init__ from RuntimeError
__str__  from BaseException

  * BBar(B, Bar):
__init__ from RuntimeError
__str__  from Bar


Finally everything make sense. And make think about be careful when
doing multiple inheritance.

Any thoughts?

~Rolando
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Easy questions from a python beginner

[Steven D'Aprano]

On Fri, 23 Jul 2010 09:22:16 -0700, Stephen Hansen wrote:

> On 7/23/10 2:05 AM, Steven D'Aprano wrote:
>> On Thu, 22 Jul 2010 21:23:05 -0700, Stephen Hansen wrote:
>> 
>>> On 7/22/10 7:47 PM, wheres pythonmonks wrote:
>> [...]
 The truth is that I don't intend to use these approaches in anything
 serious.  However, I've been known to do some metaprogramming from
 time to time.
>>>
>>> Depending on how you define "metaprogramming", Python is pretty
>>> notoriously ill-suited towards the task (more, its basically
>>> considered a feature it doesn't let you go there) -- or, it allows you
>>> to do vast amounts of stuff with its few dark magic hooks. I've never
>>> had a satisfying definition of metaprogramming that more then 50% of
>>> any group agree with, so I'm not sure which you're looking for. :)
>> 
>> I disagree strongly at your characterisation that Python is notorious
>> for being ill-suited towards metaprogramming. I'd say the complete
>> opposite -- what is considered dark and scary metaprogramming tasks in
>> other languages is considered too ordinary to even mention in Python.
> 
> I rather think you missed my point entirely.
> 
> 'Depending on how you define "metaprogramming"'
> 
> The 'Depending' was the most important part of that sentence. 

Well, I suppose some folks might like to define words any way they like, 
so that "cat" means a reptile with no legs and "lunch time" means the 
thing you do in the shower, but I prefer to stick with common 
definitions. And the common definition of metaprogramming is that it is 
"programming about programming" -- programming where the data you are 
processing is itself programming code and data structures.

http://en.wikipedia.org/wiki/Metaprogramming

I did not miss your proviso, nor did I miss the fact that there are some 
metaprogramming techniques that Python is not very good at, such as 
runtime modification of syntax. I wrote:

"It is true that other languages (like Lisp, or Forth) allow you to write 
your own syntax. That's incredibly powerful, but it also has serious 
costs. If you can define your own syntax, that means every piece of code 
you look at is potentially a different language."

But what makes you think that the OP is trying to do that? There's 
nothing in his comments to indicate that to me. Perhaps I missed 
something.



> You go on
> to talk about runtime modification of classes and the like. That, Python
> lets you do quite readily, and then go on to do crazy amounts of without
> any significant effort.
> 
> That's one definition of metaprogramming. That one Python does well.

No, it's not a *definition* of metaprogramming. It is an aspect of 
metaprogramming, in the same way that roasting food is not a definition 
of cooking, it is one out of many cooking techniques.


> The other involves things like macros, or things where you basically
> write a new sub-language in the language itself to achieve some commonly
> desired task more efficiently (or just more succinctly). That's another
> definition of metaprogramming: where its not so much structures
> (classes, etc) which are modified at runtime (which Python lets you do
> readily), but the syntax or semantics of the language itself. That
> Python isn't game for.

You can't modify Python's syntax at runtime, but you can create DSLs in 
Python quite readily provided you accept the limitation of Python's 
syntax. There was even a tutorial at PyCon about it:

http://us.pycon.org/2010/tutorials/jensen_languages/

Although he doesn't use the term "DSL" anywhere in his essay, Guido's 
essay on creating graphs in Python essentially describes a DSL for graphs:

http://www.python.org/doc/essays/graphs/

In the Python Cookbook, Paul Dubois also gives an "application-specific-
language" for graphs in a section called "Using Python Itself as a Little 
Language". Yes, it's limited to Python syntax, but it's still a DSL.

And of course there are applications like Cheetah:

http://www.cheetahtemplate.org/

Python doesn't go all the way like Forth or Lisp, but there aren't very 
many metaprogramming techniques you can't do it in.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Socket performance

[MRAB]

Navkirat Singh wrote:
Thanks for the info : ). I will look into it ! Right now I am having a 
strange problem. I am trying to use cookies and the import function 
returns an error:


I am using python 3:

from http import cookies

*importError:* No module named http

Is it my configuration or has something changed since the documentation 
was written? Sorry I might be asking too many question, I am pretty new 
to this stuff and kinda feel lost here and there : (



It works for me (Python 3.1.2).
--
http://mail.python.org/mailman/listinfo/python-list

Re: Download Microsoft C/C++ compiler for use with Python 2.6/2.7 ASAP

[Grant Edwards]

On 2010-07-24, Lawrence D'Oliveiro  wrote:
> In message , Robert Kern 
> wrote:
>
>> There are also utilities for mounting ISOs directly without burning
>> them to a physical disk.
>
> You need special utilities to do this??

Not if the OS and VFS are competently designed.  In Linux all you need
to do is this:

  mount -o loop /path/to/file.iso /mount/point

Apparently you've got to jump through all sorts of hoops using 3rd
party software to do something analgous in MS Windows.
  
-- 
Grant
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Unicode error

[Steven D'Aprano]

On Fri, 23 Jul 2010 22:46:46 +0100, Nobody wrote:

> On Fri, 23 Jul 2010 10:42:26 +, Steven D'Aprano wrote:
> 
>> Don't write bare excepts, always catch the error you want and nothing
>> else.
> 
> That advice would make more sense if it was possible to know which
> exceptions could be raised. In practice, that isn't possible, as the
> documentation seldom provides this information. Even for the built-in
> classes, the documentation is weak in this regard; for less important
> modules and third-party libraries, it's entirely absent.

Aside: that's an awfully sweeping generalisation for all third-party 
libraries.

Yes, the documentation is sometimes weak, but that doesn't stop you from 
being sensible. Catching any exception, no matter what, whether you've 
heard of it or seen it before or not, is almost never a good idea. The 
two problems with bare excepts are:

* They mask user generated keyboard interrupts, which is rude.

* They hide unexpected errors and disguise them as expected errors.

You want unexpected errors to raise an exception as early as possible, 
because they probably indicate a bug in your code, and the earlier you 
see the exception, the easier it is to debug.

And even if they don't indicate a bug in your code, but merely an under-
documented function, it's still better to find out what that is rather 
than sweep it under the carpet. You will have learned something new ("oh, 
the httplib functions can raise socket.error as well can they?") which 
makes you a better programmer, you have the opportunity to improve the 
documentation, you might want to handle it differently ("should I try 
again, or just give up now, or reset the flubbler?").

If you decide to just mask the exception, rather than handle it in some 
other way, it is easy enough to add an extra check to the except clause.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: understanding the mro (long)

[Steven D'Aprano]

On Fri, 23 Jul 2010 22:42:28 -0400, Rolando Espinoza La Fuente wrote:

> TL;DR: if you want to stay sane, don't inherit two classes that share
> same inheritance graph
> 
> I recently got puzzled by a bug from a legacy lib (ClientForm) which
> have this code:
[...]
> Finally everything make sense. And make think about be careful when
> doing multiple inheritance.
> 
> Any thoughts?


Wow. Nice work, thanks for taking the time for documenting this publicly.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Multiple versions of Python coexisting in the same OS

[Edward Diener]

Are there any documents about multiple versionsof Python coexisting in 
the same OS ( Windows in my case ) and what pitfalls to look out for ? I 
have already run into a number of them. I installed Python 2.7 and 3.1.2 
into completely folders, but immediately ran into serious problems 
executing a Python script.


The first problem is that just invoking Python will start whichever 
version is first in the PATH, and this is true from the command line or 
internally in a Python script.


The second problem is that invoking a script ( some xxx.py ) will start 
whichever version of Python is associated with the .py extension.


The third problem is if some software expects its scripts, which it puts 
in some Python subdirectory to be in the PATH.


There may be other considerations but overall having to versions 
coexisting has turned out to be a big headache involving both changes in 
the PATH and in the .py association.


Does anybody know of other things to look out for ?
--
http://mail.python.org/mailman/listinfo/python-list

Re: understanding the mro (long)

[Benjamin Kaplan]

On Fri, Jul 23, 2010 at 7:42 PM, Rolando Espinoza La Fuente
 wrote:
> TL;DR: if you want to stay sane, don't inherit two classes that share
> same inheritance graph
>
> I recently got puzzled by a bug from a legacy lib (ClientForm)
> which have this code:
>
>    class ParseError(sgmllib.SGMLParseError,
>                     HTMLParser.HTMLParseError,
>                     ):
>        pass
>
> And fails because takes __init__ from sgmllib and __str__ from HTMLParser
> where __str__ uses attributes set by HTMLParser's init.
>
> At first look, I thought was just matter to swap the inherit classes.
> But a deeper
> look take me to the python's mro reading:
> http://www.python.org/download/releases/2.3/mro/
>
> And to reproduce the error I code this:
>
> class Foo(object):
>    def __init__(self, msg):
>        self.msg = msg
>
>    def __str__(self):
>        return 'Foo: ' + self.msg
>
> class Bar(Exception):
>    def __init__(self, msg):
>        self.msg = msg
>
>    def __str__(self):
>        return 'Bar: ' + self.msg
>
> class A(Exception):
>    pass
>
> class B(RuntimeError):
>    pass
>
> class AFoo(A, Foo): pass
> class ABar(A, Bar): pass
>
> class BFoo(B, Foo): pass
> class BBar(B, Bar): pass
>
> print AFoo('ok') # ok
> print ABar('ok') # Bar: ok
>
> print BFoo('ok') # ok
> print BBar('fail') # AttributeError: ... not attribute 'msg'
>
> # EOF
>
> After running the code I was still confused. So I read carefully again
> the mro stuff. And ended doing this inheritance tree:
>
>       object (__init__, __str__)
>          |    \
>          |    Foo (__init__, __str__)
>          |
>  BaseException (__init__, __str__)
>          |
>          |
>          |
>      Exception (__init__)
>     /    |     \
>    A    |     Bar (__init__, __str__)
>          |
>  StandardError (__init__)
>          |
>          |
>          |
>    RuntimeError (__init__)
>    /
>   B
>
> Then I figure out the method resolution following the inheritance graph:
>  * AFoo(A, Foo):
>    __init__ from Exception
>    __str__  from BaseException
>
>  * ABar(A, Bar):
>    __init__ from Bar
>    __str__  from Bar
>
>  * BFoo(B, Foo):
>    __init__ from RuntimeError
>    __str__  from BaseException
>
>  * BBar(B, Bar):
>    __init__ from RuntimeError
>    __str__  from Bar
>
>
> Finally everything make sense. And make think about be careful when
> doing multiple inheritance.
>
> Any thoughts?
>

Two things:

First of all, avoid multiple inheritance if you can. It's usually
unnecessary in Python because of duck typing (unless you need to
inherit the actual behavior and not just a list of methods), and as
you've noticed, the MRO gets messy.


And second, not to in any way diminish the work you did tracing out
the inheritance tree and working through the inheritance, but Python
has easier ways of doing it :)

>>> BBar.__mro__
(, , , , , , , )
>>> '__str__' in BBar.__dict__
False
>>> '__str__' in Bar.__dict__
True
>>> for cls in BBar.__mro__ :
if '__str__' in cls.__dict__ :
print cls
break




> ~Rolando
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: A portable LISP interpreter that includes all the major list-processing functions is described. A complete, annotated listing of the program's code, written in PASCAL, is included.

[TheFlyingDutchman]

On Jul 23, 12:06 pm, Emmy Noether  wrote:
> Title   Portable LISP interpreter
> Creator/Author  Cox, L.A. Jr. ; Taylor, W.P.
> Publication Date        1978 May 31
> OSTI Identifier OSTI ID: 7017786
> Report Number(s)        UCRL-52417
> DOE Contract Number     W-7405-ENG-48
> Resource Type   Technical Report
> Research Org    California Univ., Livermore (USA). Lawrence Livermore
> Lab.
> Subject 99 GENERAL AND MISCELLANEOUS//MATHEMATICS, COMPUTING, AND
> INFORMATION SCIENCE; COMPUTER CODES; L CODES; COMPUTERS; PROGRAMMING;
> PROGRAMMING LANGUAGES
> Description/Abstract    A portable LISP interpreter that includes all the
> major list-processing functions is described. A complete, annotated
> listing of the program's code, written in PASCAL, is included.
> Country of Publication  United States
> Language        English
> Format  Medium: X; Size: Pages: 21
> Availability    Dep. NTIS, PC A02/MF A01.
> System Entry Date       2008 Feb 12

Is this available online? If only in hardcopy form, do they lend it
out?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: non-blocking IO EAGAIN on write

[Kushal Kumaran]

On Fri, Jul 23, 2010 at 2:15 PM, Thomas Guettler  wrote:
> Hi,
>
> I use non-blocking io to check for timeouts. Sometimes I get EAGAIN (Resource 
> temporarily unavailable)
> on write(). My working code looks like this. But I am unsure how many bytes 
> have been written to the
> pipe if I get an EAGAIN IOError. Up to now I retry with the same chunk.
>
> If I get EAGAIN can I just sleep, and then retry with the same data chunk?
>
> pipe=subprocess.Popen(cmd, stdin=subprocess.PIPE, bufsize=-1)
> fcntl.fcntl(pipe.stdin, fcntl.F_SETFL, os.O_NONBLOCK)
> 
> chunk_size=1024
>        while select.select([], [pipe.stdin], [], 5):

Please read the documentation of the select.select function.  You are
not using it correctly.  After the call returns, you need to check
whether the descriptor you are interested in is actually ready.  If
select reaches your timeout, it will simply return three empty lists,
and your write will get EAGAIN.  In general, after select has told you
a descriptor is ready, the first write after that should always
succeed.

BTW, from the documentation of file objects, it seems write always
wants to write the entire string you give it, since it does not return
anything.  In that case, you might want to get the descriptor for the
file object (using pipe.fileno()) and use that for writing.

> 

-- 
regards,
kushal
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: understanding the mro (long)

[Rolando Espinoza La Fuente]

On Sat, Jul 24, 2010 at 12:28 AM, Benjamin Kaplan
 wrote:
[...]
>
> And second, not to in any way diminish the work you did tracing out
> the inheritance tree and working through the inheritance, but Python
> has easier ways of doing it :)
>
 BBar.__mro__
> (, ,  'exceptions.RuntimeError'>, ,  '__main__.Bar'>, ,  'exceptions.BaseException'>, )

Yes, actually I looked at __mro__ to confirm that I was right.

 '__str__' in BBar.__dict__
> False
 '__str__' in Bar.__dict__
> True

I see! I couldn't figure out how to find if a method is defined within
given class.

 for cls in BBar.__mro__ :
>        if '__str__' in cls.__dict__ :
>                print cls
>                break
>
>
> 

This is good one! It could save time figuring out where a method comes from.
Anyway, was a good exercise to figure out the mro by hand :)

Thanks for your comments Benjamin and Steven.

~Rolando
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python as a scripting language. Alternative to bash script?

[Tim Harig]

On 2010-07-24, Lawrence D'Oliveiro  wrote:
> In message , Tim Harig wrote:
>
>> On 2010-07-05, Lawrence D'Oliveiro 
>> wrote:
>>>
>>> I???ve never come across any system where you could string together
>>> multiple GUI apps, or even multiple GUI operations in the same app, in
>>> any sensible or effective way at all. GUIs just aren???t designed to work
>>> that way.
>> 
>> You can if they are designed to be used externally.  COM is an execellent
>> example of this ...
>
> But COM is an IPC architecture, which has nothing to do with GUI apps. 

Not exactly.  IPC allows you to send and receive information from another
process; but, it doesn't give you internal access to the process.  COM is
much closer to an OO style of RPC.

> Conflating the two leads to precisely the sort of inflexibility that is 
> characteristic of GUIs. For example, things happening with the GUI on-screen 
> while a script is running that are at best distracting to the user, or at 
> worst vulnerable to breaking if the user accidentally presses a key or 
> clicks in the wrong place.

COM object have a visibility property that determines whether or not they
are displayed to the user.

>> I have worked with complex business process automation centered around
>> Excel and I have automated some really ugly AJAX style web based
>> applications that would only work inside of IE6 by accessing IE through
>> its COM interface.
>
> If those apps really were using AJAX, then why did you need to script a Web 
> browser at all? You could just make direct HTTP requests to retrieve the 
> data.

The first and primary reason was that the application server required
a client side SSL certificate.  We didn't have access to a certificate
that we could get to work Python's SSL or HTTPS modules.

The second reason was that we had considerable difficulty figuring out how
the server side interfaces worked from the slew of horrible Javascript.
You really have to have seen it to know just what a mess this really was.
Every call was literally wrapped in dozens of almost identical wrapper
functions that you had to follow to attempt to figure out what was going
on.  Parts of the rendered form which looked like text field entries really
weren't until a series of events took place to change them.  They were used
as flags indicating the state of various information.  I really can't
overstate how ugly this application really was.  

I was working on trying to figure out the serverside interface while my
partner was trying to get an open SSL connection.  By the time he gave up
trying to convert our key to something we could use, I decided it was
simpler just to use COM and internet explorer.  It worked pretty well.

>> Automating GUI applications requires interal access to the program through
>> some kind of interface ...
>
> Some kind of interface that bypasses the GUI. This is why so many Open 
> Source apps are structured as a GUI front end to a completely separate 
> command-line tool (e.g. MPlayer). That way a script can use the latter 
> without having to go through the former.

Which is great where the a separate backend exists.  That isn't the case
for a *huge* number of GUI applications; but, You don't always get to pick
what you are going to need to automate.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: understanding the mro (long)

[Michele Simionato]

On Jul 24, 4:42 am, Rolando Espinoza La Fuente 
wrote:
> Finally everything make sense. And make think about be careful when
> doing multiple inheritance.
>
> Any thoughts?
>
> ~Rolando

I am not fond of multiple inheritance either and I wrote at length
about the dangers of it. If you do not know it already, you may be
interested in reading my  "Mixins considered harmful" series
http://www.artima.com/weblogs/viewpost.jsp?thread=246341 (together
with any blog posts on super and related subjects).
-- 
http://mail.python.org/mailman/listinfo/python-list

'as' keyword - when/how to use it

[Dummey]

I am having the hardest time trying to find documentation on proper
use of the 'as' keyword (aside from . I initially thought that I would
be allowed to do something such as:

import shared.util as util

The as statement seems to be causing a lot of ''module' object has no
attribute'. Is it safe to assume that this is not the way to use it?
-- 
http://mail.python.org/mailman/listinfo/python-list