Re: If Scheme is so good why MIT drops it?

[Xah Lee]

PHP is popular because it is geared for the server-side web scripting
lang, and simpler and easy to use, than popular free alternatives at
the time (such as Perl and Java's JSP).

Python became popular primarily because its ease-to-read syntax.

Btween the two, PHP is much easier to use, and much a pleasure to
program in. Python is a pain in the ass.

PHP is functional. The language is not elegant, lots of
inconsistancies. However, it's a joy to use for any practical task in
its web scripting field. PHP has one of the best documentation among
open source computer languages, i'd say top 5.

Python is a twist, with half-assed lambda, a culty community thinking
"computer science R us", and it has this OOP obsession. The Guido guy
do not understand functional programing, but has the pleasure to
actitively badmouth it.

References:

• Language, Purity, Cult, and Deception
  http://xahlee.org/UnixResource_dir/writ/lang_purity_cult_deception.html

• What Languages to Hate
  http://xahlee.org/UnixResource_dir/writ/language_to_hate.html

• Lambda in Python 3000
  http://xahlee.org/perl-python/python_3000.html

• Python Documentation Problems
  http://xahlee.org/perl-python/python_doc_index.html

• Examples Of Quality Documentation In The Computing Industry
  http://xahlee.org/perl-python/quality_docs.html

• Xah's Perl and Python Tutorial
  http://xahlee.org/perl-python/index.html

• Xah's PHP Tutorial
  http://xahlee.org/php/index.html

  Xah
∑ http://xahlee.org/

☄
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: len() should always return something

[Steven D'Aprano]

On Fri, 24 Jul 2009 19:50:54 -0700, Dr. Phillip M. Feldman wrote:

> Here's a simple-minded example:
> 
> def dumbfunc(xs):
>for x in xs:
>   print x
> 
> This function works fine if xs is a list of floats, but not if it is
> single float.  It can be made to work as follows:
> 
> def dumbfunc(xs):
>if isinstance(xs,(int,float,complex)):
>   xs= [xs]
>for x in xs:
>   print x
> 
> Having to put such extra logic into practically every function is one of
> the annoying things about Python.


But it's not "practically every function". It's hardly any function at 
all -- in my code, I don't think I've ever wanted this behavior. I would 
consider it an error for function(42) and function([42]) to behave the 
same way. One is a scalar, and the other is a vector -- they're different 
things, it's poor programming practice to treat them identically.

(If Matlab does this, so much the worse for Matlab, in my opinion.)

However, if you want this behaviour, it's easy enough to get it with a 
decorator:

from functools import wraps
def matlab(func):
"""Decorate func so that it behaves like Matlab, that is, 
it considers a single scalar argument to be the same as a 
vector of length one."""
@wraps(func)
def inner(arg, **kwargs):
# If arg is already a vector, don't touch it.
try:
# If this succeeds, it's a vector of some type.
iter(arg)
except TypeError:
# It's a scalar.
arg = [arg]
# Now call the decorated function (the original).
return func(arg, **kwargs)
return inner

With this decorator in hand, you can now easily get the behaviour you 
want with a single extra line per function:

>>> @matlab
... def mean(numbers):
... return sum(numbers)/len(numbers)
...
>>> mean([4.5])
4.5
>>> mean(4.5)
4.5
>>> mean([4.5, 3.6])
4.0498


Decorators are extremely powerful constructs, and well worth learning. As 
an example, here's a similar decorator that will accept multiple 
arguments, like the built-in functions min() and max():

def one_or_many(func):
"""Decorate func so that it behaves like the built-ins
min() and max()."""
@wraps(func)
def inner(*args, **kwargs):
# If we're given a single argument, and it's a vector, 
# use it as args.
if len(args) == 1:
try:
iter(args[0])
except TypeError:
pass
else:
# No exception was raised, so it's a vector.
args = args[0]
# Now call the decorated function (the original).
return func(args, **kwargs)
return inner


And then use it:

>>> @one_or_many
... def minmax(numbers):
... """Return the minimum and maximum element of numbers, 
... making a single pass."""
... # the following will fail if given no arguments
... smallest = biggest = numbers[0]  
... for x in numbers[1:]:
... if x < smallest:
... smallest = x
... elif x > biggest:
... biggest = x
... return (smallest, biggest)
...
>>>
>>> minmax([2, 4, 6, 8, 1, 3, 5, 7])
(1, 8)
>>> minmax(2, 4, 6, 8, 1, 3, 5, 7)
(1, 8)




-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: If Scheme is so good why MIT drops it?

[Tayssir John Gabbour]

On Jul 24, 11:58 pm, ACL  wrote:
> I actually think that the thing holding lisp back is 'bus factor'.
>
> Lets assume I have a java project and a lisp project:
>
> Java project:
> I have maybe 10 or 12 people on my team working on various subsystems
> of my project. There are probably one or two 'technical leader' types
> in the group, and a bunch of others who are sort of 'serfs', banging
> out java classes. Lets say one of my important guys gets totally
> splattered by a bus... I've still got another one left! I can rely on
> the other guy to keep things afloat while I train up a new leader
> type.
>
> Lisp project:
> I don't need as many people. I have 3 or 4 people, and one person is
> my technical leader and lisp guru. Guru is probably in charge of more
> difficult macros and (because of that), also in charge of the overall
> design (macros are design patterns embodied in code). Lets say he gets
> totally annihilated by the bus. What do I do now? I had all my eggs in
> one basket and my project is now stalled.

A Clojure programmer mentioned interesting solutions to this used by
his company, such as partnerships with similarly Agile companies.
http://programmingtour.blogspot.com/2009/07/conversation-with-stuart-halloway.html

I agree that the bus factor -- as well as some other problems -- works
against Lisp's acceptance.


All the best,
Tayssir
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: len() should always return something

[Hendrik van Rooyen]

On Friday 24 July 2009 16:45:40 Mark Dickinson wrote:

> On Jul 24, 3:11 pm, "Rhodri James" 
>
> wrote:
> > Which doesn't make your point less valid.  In fact I'd go so
> > far as to argue that what len() gives you is the number of
> > items in a container, so len(7) should return 0.
>
> Nah.  7 contains three bits, so len(7) should *clearly* return 3.

Almost right - however the first seven that you typed was a string
seven and not an int - so the hex is 37 and that takes at least
six bits...

:-)

Now if only someone can figure out a reason why it should
return seven, then it would be perfect!

- Hendrik
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: len() should always return something

[Carl Banks]

On Jul 23, 11:35 pm, "Dr. Phillip M. Feldman" 
wrote:
> Some aspects of the Python design are remarkably clever, while others leave
> me perplexed. Here's an example of the latter: Why does len() give an error
> when applied to an int or float? len() should always return something; in
> particular, when applied to a scalar, it should return a value of 1. Of
> course, I can define my own function like this:
>
> def mylen(x):
>    if isinstance(x,int) or isinstance(x,float): return 1
>    return len(x)
>
> But, this shouldn't be necessary.

I knew that you were coming from Matlab as soon as I saw the subject
line.

I use both Matlab and Python very often, and I understand the design
decisions behind both (insofar as Matlab can be considered a
"design").

The thing to keep in mind about Python is that it's a general purpose
language, not just for mathematical computation, and there are some
things in Python that are a bit suboptimal for math calculations.
This is one of them: when the types of objects you are dealing with
most often are numbers, then there's not much chance for ambiguity
when treating a scalar as a 1x1 matrix.

However, when doing general purpose programming, you often use lots of
types besides numbers, some of which are containers themselves.  This
creates the possibility of ambiguity and inconsistent behavior.  Let's
consider your example function.

def dumbfunc(xs):
   for x in xs:
  print x

You can use it on an list, like this, and there is no ambiguity:

dumbfunc([1,2,3])

Now suppose Python allowed numbers to be treated as degenerate
sequences of length 1.  Then you could pass a number, and it would
work as you expect:

dumbfunc(1)

However, because Python is general purpose, you might also want to
pass other types around, such as strings.  Say you wanted to print a
list of string, you would do this, and it would work as expected:

dumbfunc(["abc","def","ghi"])

Now, the problem.  In the numbers example, you were able to treat a
single number as a degenerate list, and by our supposition, it
worked.  By analogy, you would expect to be able to do the same with a
list of strings, like this:

dumbfunc("abc")

Whoops: doesn't work.  Instead of printing one string "abc" it prints
three strings, "a", "b", and "c".

By allowing scalar numbers to act as degenerate lists, you've
introduced a very bad inconsistency into the language, you've made it
difficult to learn the language by analogy.

For a general purpose language there is really no second-guessing this
design decision.  It would be unequivocally bad to have atomic types
act like sequences.  The reason it isn't so bad in Matlab is that you
can rely on the elements of an array to be scalar.

Matlab, for its part, does contain some types (cell arrays and
structures) that can contain non-scalars for cases when you want to do
that.  However, you will notice that no scalar will ever be treated as
a degenerate cell array or structure.  Even a poorly designed, ad hoc
language like Matlab won't try to treat scalar numbers as cell arrays,
because ambiguity like I described above will appear.  Nope, it takes
an abomination like Perl to do that.

As for how I would recommend you translate the Matlab function that
rely on this: you need to decide whether that scalar number you are
passing in is REALLY a scalar, or a degenerate vector.  If it's the
latter, just get over it and pass in a single-item list.  I have done
these kinds of conversions myself before, and that's the way to do
it.  Trying to be cute and writing functions that can work with both
types will lead to trouble and inconsistency, especially if you are a
Python newbie.  Just pass in a list if your function expects a list.


Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: len() should always return something

[Diez B. Roggisch]

Dr. Phillip M. Feldman schrieb:

Here's a simple-minded example:

def dumbfunc(xs):
   for x in xs:
  print x

This function works fine if xs is a list of floats, but not if it is single
float.  It can be made to work as follows:

def dumbfunc(xs):
   if isinstance(xs,(int,float,complex)): xs= [xs]
   for x in xs:
  print x

Having to put such extra logic into practically every function is one of the
annoying things about Python.


And where comes "len(xs)" into play here? What you want is iteration 
over scalars.


I do think that if you frequently have to write code like that, you are 
writing errorneous code.


But might that as it is, a simple


def iterable(i):
if isinstance(i, (int, float, complex)):
   return [i]
return i


is all you need. So you can write


for x in iterable(xs):


wherever you expect values to be either scalar or iterable.

Diez
--
http://mail.python.org/mailman/listinfo/python-list

Re: len() should always return something

[Piet van Oostrum]

> Steven D'Aprano  (S) wrote:

>S> Chris, I'm curious why you think that these Zen are relevant to the OP's 
>S> complaint.

>S> Re explicit vs implicit, len(42) is just as explicit as len([42, 23]).

>S> Arguably (I wouldn't argue this, but some people might) ints aren't 
>S> "special enough" to break the rule that len(obj) should always return 
>S> something.

>S> (I don't actually agree, but some people might be able to produce a 
>S> coherent argument why len() should apply equally to all objects.)

>S> Re errors passing silently, the OP doesn't believe that len(42) should be 
>S> an error, so that's not relevant.

>S> And there's nothing ambiguous about len(42).

len(42) should be 7.5 million.
-- 
Piet van Oostrum 
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: p...@vanoostrum.org
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How do I generate dia diagrams from python source code?

[Gabriel Genellina]

En Fri, 24 Jul 2009 17:52:47 -0300, Qauzzix  escribió:


Since I have been using dia to make my UML diagrams. I also found an
util named dia2code that generates python code from dia diagram. Now
that I have that option I really want to find a way to generate dia
diagram from existing code and/or maintain my diagrams.

I have been googling  like crazy trying to find a way but with no
luck. Anyone here know how to do this?


SPE does that: http://pythonide.stani.be/

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: Help understanding the decisions *behind* python?

[Hendrik van Rooyen]

On Friday 24 July 2009 17:07:30 Inky 788 wrote:
> On Jul 23, 3:42 am, Hendrik van Rooyen 
>
> wrote:
8<
> > Steven showed why you cannot have a mutable thing
> > as a key in a dict.
> >
> > if you think it is contrived, then please consider how you would
> > keep track of say the colour of a pixel on a screen at position
> > (x,y) - this is about the simplest "natural" tuple format and
> > example.
>
> My guess is that this is probably the way most people do it:
>
> 
> #!/usr/bin/env python
>
> import sys
> import random
>
> if len( sys.argv ) != 3:
> print "Please pass exactly 2 ints. Exiting."
> sys.exit(1)
>
> NUM_COLUMNS = int( sys.argv[1] )
> NUM_ROWS= int( sys.argv[2] )
>
> print "Making array of %s columns by %s rows." % (NUM_COLUMNS,
> NUM_ROWS)
>
> def rand():
> return int( 255 * random.random())
>
> def make_a_pixel():
> #   red green   blue
> return [rand(), rand(), rand()]
>
> def make_a_row(num_columns):
> temp_row = []
> for i in range(num_columns):
> temp_row.append( make_a_pixel() )
> return temp_row
>
> def make_array_of_pixels(num_columns, num_rows):
> rows = []
> for i in range(num_rows):
> rows.append( make_a_row(num_columns) )
> return rows
>
> def show_pixels(pixel_array):
> for row in pixel_array:
> for pixel in row:
> print pixel, '  ',
> print
>
>
> rows_of_pixels = make_array_of_pixels(NUM_COLUMNS, NUM_ROWS)
>
> show_pixels(rows_of_pixels)
> 

Good grief! - Not a dictionary in sight!

I had something in mind where point was an (x,y) tuple,
and point_colours was a dict of possibly named points 
with a list of RGB values. 
(The colour can change, the point is fixed)

If I were doing it your way, I would use the Array module's Array.
But then, strictly speaking it would not be your way any more,
would it?

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: len() should always return something

[Hendrik van Rooyen]

On Friday 24 July 2009 21:04:55 Roy Smith wrote:


> Compressing strings to a single bit is easy.  It's the uncompressing that's
> tricky.

Not really - all you have to do is to apply the EXACT same sequence
of operations that compressed it, in reverse.

The unfortunate part is that this information is in almost all cases
larger than the original, uncompressed string, so that it kind of defeats
the object of compression.

So yes - tricky!

:-)

- Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: len() should always return something

[Hendrik van Rooyen]

On Friday 24 July 2009 22:09:15 Marcus Wanner wrote:

> First one to correctly decompress the value 0 into an ASCII character
> wins the title of the world's most capable hacker :p

that is easy.

the xor of 0 and 1 is 1, which is ASCII soh, if I remember right.

soh is start of header.

Burroughs poll select, anyone?

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: strange python scripting error

[Mark Tarver]

On 24 July, 15:45, nn  wrote:
> On Jul 23, 7:03 pm, Dave Angel  wrote:
>
>
>
>
>
> > Mark Tarver wrote:
> > > I have a very strange error.  I have two test python files test.py and
> > > python.py which contain the following code
>
> > > #!/usr/bin/python
> > > print "Content-type: text/html"
> > > print
> > > print ""
> > > print "Hello, Linux.com!"
> > > print ""
>
> > > One file (test.py) works; you call it up and it shows a web page with
>
> > > Hello, Linux.com
>
> > > The other fails with a server configuration error.  Both are running
> > > under Linux, same server, same permissions.  Running a character scan
> > > shows that both files contain the same printable characters and are
> > > therefore typographically identical.   They are absolutely the same.
>
> > > The only hint at a difference I can see is that my ftp program says
> > > the files are of unequal lengths.  test.py is 129 bytes long.
> > > python.py 134 bytes long.
>
> > > A zipped folder containing both files is at
>
> > >www.lambdassociates.org/weird.zip
>
> > > Any ideas welcome.
>
> > > Mark
>
> > Easiest explanation is that python.py has Windows-style newlines.  In
> > other words, each line ends with 0d0a, rather than the Unix convention
> > of 0a.
>
> > If your server is Unix-based, it can't handle that first line, since it
> > has an illegal character (0d) following the
>
> > #!/usr/bin/python
>
> > line.  Convert it to Unix line-endings.
>
> > DaveA
>
> Use dos2unix for conversion of the longer file and try again:
>
> http://linux.about.com/od/commands/l/blcmdl1_dos2uni.htm- Hide quoted text -
>
> - Show quoted text -

That sounds the ticket - but is there anything that runs under Windows
to do the trick?

Mark
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: strange python scripting error

[Mark Tarver]

On 25 July, 10:30, Mark Tarver  wrote:
> On 24 July, 15:45, nn  wrote:
>
>
>
>
>
> > On Jul 23, 7:03 pm, Dave Angel  wrote:
>
> > > Mark Tarver wrote:
> > > > I have a very strange error.  I have two test python files test.py and
> > > > python.py which contain the following code
>
> > > > #!/usr/bin/python
> > > > print "Content-type: text/html"
> > > > print
> > > > print ""
> > > > print "Hello, Linux.com!"
> > > > print ""
>
> > > > One file (test.py) works; you call it up and it shows a web page with
>
> > > > Hello, Linux.com
>
> > > > The other fails with a server configuration error.  Both are running
> > > > under Linux, same server, same permissions.  Running a character scan
> > > > shows that both files contain the same printable characters and are
> > > > therefore typographically identical.   They are absolutely the same.
>
> > > > The only hint at a difference I can see is that my ftp program says
> > > > the files are of unequal lengths.  test.py is 129 bytes long.
> > > > python.py 134 bytes long.
>
> > > > A zipped folder containing both files is at
>
> > > >www.lambdassociates.org/weird.zip
>
> > > > Any ideas welcome.
>
> > > > Mark
>
> > > Easiest explanation is that python.py has Windows-style newlines.  In
> > > other words, each line ends with 0d0a, rather than the Unix convention
> > > of 0a.
>
> > > If your server is Unix-based, it can't handle that first line, since it
> > > has an illegal character (0d) following the
>
> > > #!/usr/bin/python
>
> > > line.  Convert it to Unix line-endings.
>
> > > DaveA
>
> > Use dos2unix for conversion of the longer file and try again:
>
> >http://linux.about.com/od/commands/l/blcmdl1_dos2uni.htm-Hide quoted text -
>
> > - Show quoted text -
>
> That sounds the ticket - but is there anything that runs under Windows
> to do the trick?
>
> Mark- Hide quoted text -
>
> - Show quoted text -

OK, got a version

http://www.bastet.com/

has dos2unix.exe for Windows.  And it solves the problem.

Many thanks all for help on this baffling error

Mark
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: If Scheme is so good why MIT drops it?

[Jon Harrop]

ACL wrote:
> Lisp project:
> I don't need as many people...

Is there any actual evidence of that?

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?u
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: non-owning references?

[Piet van Oostrum]

> "Rhodri James"  (RJ) wrote:

>RJ> The point was, and remains, that this newsgroup gets regular traffic
>RJ> from people who expect Python's variables to act like C's variables,
>RJ> demonstrating that describing them as "quite compatible" is somewhat
>RJ> misleading.

So let' study these postings carefully to see what the actual confusion
is. I myself think that the confusion is not so much the variable
concept but the vale vs. reference semantics of objects. As such the
same thing plays a role in Java. Although the binding mechanisms in Java
and Python are different I don't think the actual semantics are so much
difference as to cause much confusion (with C it could be more). But
such a study could reveal that.

BTW. Which postings from (say) the last month did you have in mind?
-- 
Piet van Oostrum 
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: p...@vanoostrum.org
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: strange python scripting error

[Dave Angel]

Mark Tarver wrote:


  



Use dos2unix for conversion of the longer file and try again:
  
http://linux.about.com/od/commands/l/blcmdl1_dos2uni.htm-Hide quoted text -
  
- Show quoted text -
  

That sounds the ticket - but is there anything that runs under Windows
to do the trick?

Mark- Hide quoted text -

- Show quoted text -



OK, got a version

http://www.bastet.com/

has dos2unix.exe for Windows.  And it solves the problem.

Many thanks all for help on this baffling error

Mark

  
I use metapad for simple text editing, see
http://en.wikipedia.org/wiki/Metapad


It's free, loads very quickly, and has a number of features I like.  One 
of its trivial features is to read/write files in four different 
formats.  In the file menu, you just choose what file-format you need now.


Another thing I'd point out is that some ftp programs will do this 
conversion as the file is being sent between a local DOS machine and a 
Unix machine on the internet.



DaveA

--
http://mail.python.org/mailman/listinfo/python-list

Re: how can a child thread notify a parent thread its status?

[Piet van Oostrum]

> davidj411  (d) wrote:

>d> could i see an example of this maybe?

queue is a shared Queue instance.

The parent puts the work in the queue with queue.put(work_object).
After all work has been put in the queue it puts a sentinel

queue.put(None)

Each child thread (consumer) has a loop getting things out of the queue:

while True:
work = queue.get()
if work is None: break
#do real stuff using work

queue.put(None) #put back the sentinel for other worker threads.

#finish

(Thanks to Dennis Lee Bieber)

Instead of None you can also create a special sentinel object and use that.
-- 
Piet van Oostrum 
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: p...@vanoostrum.org
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: cgi.fieldstorage()

[gert]

On Jul 25, 2:33 am, "Diez B. Roggisch"  wrote:
> gert schrieb:
>
> > On Jul 24, 7:32 pm, "Diez B. Roggisch"  wrote:
> >> gert schrieb:
>
> >>> this is a non standard way to store multi part post data on disk
> >>> def application(environ, response):
> >>>     with open('/usr/httpd/var/wsgiTemp','w') as f:
> >>>         while True:
> >>>             chunk = environ['wsgi.input'].read(8192).decode('latin1')
> >>>             if not chunk: break
> >>>             f.write(chunk)
> >>>     response('200 OK',[])
> >>>     return ['complete']
> >>> my question is how do i handle the file, so i can shuffle it into a db
> >>> using small chunks of memorie ?
> >> I don't think that's possible with the current DB-API. There is no
> >> stream-based BLOB-interface (as e.g. JDBC offers).
>
> >> So the answer certainly depends on your used RDBMS. For oracle, you
> >> would be lucky:
>
> >>http://cx-oracle.sourceforge.net/html/lob.html
>
> >> Other adapters I don't know about.
>
> > sqlite :) ok let say for now it would be impossible on a db level, but
> > before i reach the impossible, i still need to parse the file to
> > prepare the chunks. How do i do that ? How do i get the chunks without
> > loading the hole file into memorie ?
>
> It's memory - memorie might be some nice dutch girl you know :)
>
> Apart from that, your code above does exactly that - reads the data
> chunkwise. If the WSGI-implementation works proper, this will be the
> socket's input stream, so there is no memory overhead involved.
>
> Now of course if you want to have a multi-pass processing of the file
> without putting it into memory, then you need to save it to the harddisk
> befor.
>
> But honestly - we are talking about a web-application here. My
> DSL-connection has 1 MBit upstream, the average server has at least 2GB
> of memory available - so we are talking 2 seconds uploading time to
> fill that memory. Which is about 5 hours. And you are decoding the data
> to a certain decoding, so we are not talking about binary data here -
> are you really sure memory is an issue?
>

What about some http upload resume features ?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: non-owning references?

[Carl Banks]

On Jul 24, 8:14 am, Ben Finney  wrote:
> Hrvoje Niksic  writes:
> > The term "variable" is used in the Python language reference and
> > elsewhere
>
> Yes. It should also be abundantly clear from the constant stream of
> confused newbies on this point that its usage of that term is different
> to what many expect from usage elsewhere.

Personally, I haven't noticed a constant stream, just a few here and
there.

I expect that someone used to C variables will expect Python variables
to behave the same way even if you call them something different, so
what really is the point?


Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: If Scheme is so good why MIT drops it?

[Carl Banks]

On Jul 24, 11:54 pm, Xah Lee  wrote:
[snip]
> References:
>
> • Language, Purity, Cult, and Deception
>  http://xahlee.org/UnixResource_dir/writ/lang_purity_cult_deception.html
>
> • What Languages to Hate
>  http://xahlee.org/UnixResource_dir/writ/language_to_hate.html
>
> • Lambda in Python 3000
>  http://xahlee.org/perl-python/python_3000.html
>
> • Python Documentation Problems
>  http://xahlee.org/perl-python/python_doc_index.html
>
> • Examples Of Quality Documentation In The Computing Industry
>  http://xahlee.org/perl-python/quality_docs.html
>
> • Xah's Perl and Python Tutorial
>  http://xahlee.org/perl-python/index.html
>
> • Xah's PHP Tutorial
>  http://xahlee.org/php/index.html
>
>   Xah
> ∑http://xahlee.org/

Wow, you leave no stone unturned.


> "computer science R us"

I'm stealing this.


Carl BAnks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: len() should always return something

[Marcus Wanner]

On 7/25/2009 5:34 AM, Hendrik van Rooyen wrote:

On Friday 24 July 2009 22:09:15 Marcus Wanner wrote:


First one to correctly decompress the value 0 into an ASCII character
wins the title of the world's most capable hacker :p


that is easy.

the xor of 0 and 1 is 1, which is ASCII soh, if I remember right.

soh is start of header.

Burroughs poll select, anyone?

- Hendrik


nope, not soh.

Marcus
--
http://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't `from pkg import mod' work after `del pkg.mod'?

[Piet van Oostrum]

> ryles  (r) wrote:

>r> According to http://www.python.org/doc/essays/packages.html:
>r> "The import statement first tests whether the item is defined in the
>r> package; if not, it assumes it is a module and attempts to load it."

>r> However, I've noticed that once a module is imported using the
>r> `from pkg import mod' syntax, if its name is deleted from the
>r> package namespace then subsequent imports with `from' will fail.

This is incorrectly stated. Also on the initial import it will fail, not
just on subsequent imports.

piet$ python
Python 2.6.2 (r262:71600, Apr 16 2009, 09:17:39) 
[GCC 4.0.1 (Apple Computer, Inc. build 5250)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from pkg import _impl
_impl # this is from a print statement in _impl to show that it did import
Traceback (most recent call last):
  File "", line 1, in 
ImportError: cannot import name _impl
>>> 

According to the documentation
 the statement
from pkg import _impl

_temp = __import__('pkg', globals(), locals(), ['_impl'], -1)
_impl = _temp._impl

This fails because _temp (the imported module) doesn't have a binding
for _impl because you deleted it. By the way, if you have a `normal'
package that doesn't do anything with the name _impl, after the import
the pkg module namespace will have got a binding for _impl although it is not
in your code. It will have been put there by the import code.
In the following example pkg2/__init__.py just contains the line 
A = 2

piet$ python
Python 2.6.2 (r262:71600, Apr 16 2009, 09:17:39) 
[GCC 4.0.1 (Apple Computer, Inc. build 5250)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from pkg2 import _impl
_impl
>>> import sys
>>> dir(sys.modules['pkg2'])
['A', '__builtins__', '__doc__', '__file__', '__name__', '__package__', 
'__path__', '_impl']
>>> 

It is not clear to me, however what the order is of the statements in
__init__.py and the insertion of _impl in the module namespace.
-- 
Piet van Oostrum 
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: p...@vanoostrum.org
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: len() should always return something

[Steven D'Aprano]

On Sat, 25 Jul 2009 10:03:58 +0200, Piet van Oostrum wrote:

>>S> And there's nothing ambiguous about len(42).
> 
> len(42) should be 7.5 million.

And "I don't understand your reasoning".upper() should be "Millennium 
Hand and Shrimp!".

Every function would be ambiguous if we treat it as returning an 
arbitrary result. But we don't -- semantics do matter. We expect that 
string.upper() should return the string converted to uppercase, and 
that's not ambiguous because we already know what uppercase means. Even 
if somebody didn't know what uppercase meant, they would expect that it 
had a meaning, and it was just a matter of learning what it meant: for 
example, one might take string.upper() as any of:

* convert every letter a..z to A..Z

* convert the first letter of each word to A..Z

* throw away the argument and return the literal "upper"

* count how many characters are in A..Z

and so forth.

These are some of the infinite number of *possible* meanings to 
string.upper(), but only one of them is the *actual* meaning. The method 
isn't ambiguous, because the language designers have chosen one meaning, 
and having learned what that meaning is, it's the *only* meaning you can 
legitimately use. If you (generic you) are uncertain what that meaning 
is, that's uncertainty, not ambiguity.

Similarly, we might be uncertain about the meaning of len(42) -- 
reasonable values it might return are 0, 1, 2, the number of bits in 42, 
and possibly even 7.5 million as you suggest. But that's our ignorance, 
due to the fact that we haven't yet agreed on a meaning to len(42), and 
not ambiguity: having made up our mind what len(42) should mean, it could 
only mean one thing.

However, mixed string/integer addition is a good example of ambiguity. We 
can add strings:

"1" + "2" => "12"

and we can add integers:

1 + 2 => 3

but it is ambiguous as to what "1" + 2 should return. Guido might have 
made Python behave like Perl, and define addition of a string and an int, 
but there would still be ambiguity in the expression, because we can't 
tell if the coder intended "1"+"2" and forgot to quote the two, or 
intended 1+2 and forgot to convert the string "1" to an int.

Ambiguity essentially boils down to being unable to reasonably predict 
the expectation of the coder. I say "reasonably", because if you allow 
unreasonable situations, everything is "ambiguous":

I typed:

x = 42 + y

but maybe I intended to import the math module, therefore the + operator 
is ambiguous. Reasonable? No, because we're allowed to assume the coder 
is rational, and mistakes are small.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: non-owning references?

[Steven D'Aprano]

On Sat, 25 Jul 2009 04:45:48 -0700, Carl Banks wrote:

> On Jul 24, 8:14 am, Ben Finney  wrote:
>> Hrvoje Niksic  writes:
>> > The term "variable" is used in the Python language reference and
>> > elsewhere
>>
>> Yes. It should also be abundantly clear from the constant stream of
>> confused newbies on this point that its usage of that term is different
>> to what many expect from usage elsewhere.
> 
> Personally, I haven't noticed a constant stream, just a few here and
> there.
> 
> I expect that someone used to C variables will expect Python variables
> to behave the same way even if you call them something different, so
> what really is the point?

From time to time we get newbies to Python assuming that len(list) is 
O(N), because the lists they learned about in Comp Sci 101 (or Lisp) are 
linked lists and traversing a linked list to count the items is O(N). Do 
you believe that if Python lists were called "arrays", or "waskilators", 
they would make the same mistake?

[Generalisation] 
C programmers have a mental model for how "variables" behave. They have 
no such mental model for how "objects bound to names" behave, and so they 
are less likely to jump to conclusions about the behaviour of name-
bindings. So I think you are mostly wrong.

However, only mostly -- human beings are generalisers par excellence, or 
rather, *over*-generalisers par excellence. Experienced coders are likely 
to rapidly realise that name-binding is very similar to the C variable 
model, and some percentage of them will over-generalise to assume that 
name-binding is the same as the C model. But if they do, at least that 
will be their fault for jumping to conclusions, not our fault for 
misleading them.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: len() should always return something

[Piet van Oostrum]

> Steven D'Aprano  (SD) wrote:

>SD> Ambiguity essentially boils down to being unable to reasonably predict 
>SD> the expectation of the coder. I say "reasonably", because if you allow 
>SD> unreasonable situations, everything is "ambiguous":

That's for me the reason that len(42) is ambiguous. The OP apparently
had 1 as expectation, whereas my first thought was the minimal number of
bits to represent the number and 7.5 million came later :=). The number
of bits I certainly find reasonable, and I would find the number of
decimal digits equally reasonable. More so than 1, actually. 1 as the
length of an int doesn't give any information.
-- 
Piet van Oostrum 
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: p...@vanoostrum.org
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: comments? storing a function in an object

[Aahz]

In article <20efdb6a-c1a5-47dc-8546-7c4ae548e...@g1g2000pra.googlegroups.com>,
Carl Banks   wrote:
>On Jul 22, 8:38=A0pm, a...@pythoncraft.com (Aahz) wrote:
>> In article .com>,
>> Carl Banks =A0 wrote:
>>>
>>>You have to be REALLY REALLY careful not to pass any user-supplied
>>>data to it if this is a server running on your computer, of course.
>>
>> Unless, of course, your users are paying for this service.
>
>Well, yes, but I assume that by the time you're deliberately letting
>users pay to run their programs on your server, you will already have
>deployed a full-blown, multi-tiered security strategy that includes
>validation by the server process.  That was sort of beyond the scope
>of the OP's question.

That's not necessarily a good assumption.
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"At Resolver we've found it useful to short-circuit any doubt and just
refer to comments in code as 'lies'. :-)"
--Michael Foord paraphrases Christian Muirhead on python-dev, 2009-03-22
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python fast HTML data extraction library

[Aahz]

In article <37da38d2-09a8-4fd2-94b4-5feae9675...@k1g2000yqf.googlegroups.com>,
Filip   wrote:
>
>I tried to fix that with BeautifulSoup + regexp filtering of some
>particular cases I encountered. That was slow and after running my
>data scraper for some time a lot of new problems (exceptions from
>xpath parser) were showing up. Not to mention that BeautifulSoup
>stripped almost all of the content from some heavily broken pages
>(50+KiB page stripped down to some few hundred bytes). Character
>encoding conversion was a hell too - even UTF-8 pages had some non-
>standard characters causing issues.

Have you tried lxml?
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"At Resolver we've found it useful to short-circuit any doubt and just
refer to comments in code as 'lies'. :-)"
--Michael Foord paraphrases Christian Muirhead on python-dev, 2009-03-22
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: missing 'xor' Boolean operator

[Albert van der Horst]

In article ,
Jean-Michel Pichavant   wrote:
>Christian Heimes wrote:
>> Chris Rebert wrote:
>>
>>> Using the xor bitwise operator is also an option:
>>> bool(x) ^ bool(y)
>>>
>>
>> I prefer something like:
>>
>> bool(a) + bool(b) == 1
>>
>> It works even for multiple tests (super xor):
>>
>>   if bool(a) + bool(b) + bool(c) + bool(d) != 1:
>>   raise ValueError("Exactly one of a, b, c and d must be true")
>>
>> Christian
>>
>>
>While everyone's trying to tell the OP how to workaround the missing xor
>operator, nobody answered the question "why is there no xor operator ?".
>
>If the question was "Why is there no 'or' operator ?", would "because A
>or B <=> not(not A and not B)" be a proper answer ?

No. I think it is because and/or can be extended to be sensible
in a context where objects can be used. (What others have expressed
as having short-circuit evaluation. So sce indeed is the underlying
reason that and/or can be extended sensibly to objects.)

Remains whether we need an xor that only works and requires that
both operands are booleans. That one we have already!
It is called != .

(a!=b)!=c
and
a!=(b!=c)

are the same for booleans, so can indeed be expressed
a!=b!=c   (associativy of xor)

>
>JM
>
Groetjes Albert

--
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
alb...@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: len() should always return something

[Hendrik van Rooyen]

On Saturday 25 July 2009 14:59:43 Steven D'Aprano wrote:
> On Sat, 25 Jul 2009 10:03:58 +0200, Piet van Oostrum wrote:
> >>S> And there's nothing ambiguous about len(42).
> >
> > len(42) should be 7.5 million.
>
> And "I don't understand your reasoning".upper() should be "Millennium
> Hand and Shrimp!".

That does it.
I will be kinder to you in future.
Anybody who quotes foul old Ron cannot be all bad.

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: strange error when trying to log something

[Aahz]

In article ,
Peter Otten  <__pete...@web.de> wrote:
>
>I have a hunch that you are triggering a reload() somewhere. Example:
>
>Python 2.6.2 (release26-maint, Apr 19 2009, 01:58:18)
>[GCC 4.3.3] on linux2
>Type "help", "copyright", "credits" or "license" for more information.
 import weakref
 weakref.WeakValueDictionary()
>
 import UserDict
 reload(UserDict)
>
 weakref.WeakValueDictionary()
>Traceback (most recent call last):
>  File "", line 1, in 
>  File "/usr/lib/python2.6/weakref.py", line 51, in __init__
>UserDict.UserDict.__init__(self, *args, **kw)
>TypeError: unbound method __init__() must be called with UserDict instance 
>as first argument (got WeakValueDictionary instance instead)

Nice sleuthing!  How did you figure that out?
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"At Resolver we've found it useful to short-circuit any doubt and just
refer to comments in code as 'lies'. :-)"
--Michael Foord paraphrases Christian Muirhead on python-dev, 2009-03-22
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why aren't OrderedDicts comparable with < etc?

[Albert van der Horst]

In article ,
Jack Diederich   wrote:
>On Mon, Jul 20, 2009 at 10:00 AM, Steven
>D'Aprano wrote:
>> On Mon, 20 Jul 2009 09:34:24 +, Sion Arrowsmith wrote:
>>
>>> Terry Reedy =A0 wrote:
Sion Arrowsmith wrote:
> Jack Diederich =A0 wrote:
>> It isn't an OrderedDict thing, it is a comparison thing. =A0Two regul=
>ar
>> dicts also raise an error if you try to LT them.
> Python 2.5.2
 d1 =3D dict((str(i), i) for i in range (10)) d2 =3D dict((str(i), i=
>)
 for i in range (20)) d1 < d2
> True
Try reversing the definitions of d1 and d2. The dicts are probably being
compared by id (address), which is the 2.x CPython default.
>>>
>>> Like this?
>>>
>> d1 =3D dict((str(i), i) for i in range (20))
>> d2 =3D dict((str(i), i) for i in range (10))
>> d1 < d2
>>> False
>> id(d1) < id(d2)
>>> True
>>>
>>> I didn't know that comparison for anything other than equality defaulted
>>> to using id. That really is rather broken, and I'm glad 3.0 fixed it.
>>
>>
>> I don't think comparisons other than equality use id. That would be
>> rather insane. If anyone can demonstrate such comparisons in a built-in
>> or standard library class, I'd like to see it.
>>
>>
>> For the record, dicts have been comparable with < and > since at least
>> Python 1.5:
>>
>> $ python1.5
>> Python 1.5.2 (#1, Apr =A01 2009, 22:55:54) =A0[GCC 4.1.2 20070925 (Red Ha=
>t
>> 4.1.2-27)] on linux2
>> Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>
> {1: 'a', 2: 'b'} < {2: 'b', 3: 'a'}
>> 1
>>
>>
>> Reading the docs:
>>
>> http://docs.python.org/library/stdtypes.html#comparisons
>> http://docs.python.org/library/stdtypes.html#mapping-types-dict
>>
>> I can only suggest that dicts compare in an arbitrary but consistent
>> fashion.
>
>I should have specified 3.x but since we were talking about
>OrderedDicts (only in 3.1) I didn't say it explicitly.   Earlier
>versions of python were very permissive with comparisons -- it was
>rare that any cmp() raised an error and the ultimate fallback was on
>the object's id.  This is one of the non-backwards compatible changes
>in 3k.  Now comparing two of the same thing that don't have an obvious
>ordering is an error.  Is a dict "greater than" if it has a larger
>size?  if its max key is larger?  what does "max key" mean when the
>keys aren't even comparable?.  Comparing things that aren't extremely
>similar is an error now also.

With regard to < and > you are right.
But I think there is a sensible == w.r.t. dict's.
It is to mean that for each key   dict1(key) == dict2(key)
(implying that their key set must be the same)

[I could have used that for one of the euler problems.
You have a 4 by 4 field containing a red or blue square.
That is naturally a mapping of (1,1) ..(4,4) tuples to one
of the objects `blue' `red'. After moving a square you
want to know whether this is a map you already have encountered.]

>
>-Jack


--
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
alb...@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: missing 'xor' Boolean operator

[Terry Reedy]

Albert van der Horst wrote:


Remains whether we need an xor that only works and requires that
both operands are booleans. That one we have already!
It is called != .

(a!=b)!=c
and
a!=(b!=c)

are the same for booleans, so can indeed be expressed
a!=b!=c   (associativy of xor)


Not in Python

>>> a,b,c = True, False, True
>>> (a!=b)!=c
False
>>> a!=(b!=c)
False
>>> a!=b!=c
True
>>> (a!=b) and (b!=c)
True

In Math and Python, a!= is a comparison operator like <, not an arithmetic operator like + 
(or logical xor). If one has 0/1 or True/False values, we have 
arithmetic xor already, ^, which works as expected.


>>> (a^b)^c
False
>>> a^b^c
False

Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: how can a child thread notify a parent thread its status?

[scriptlear...@gmail.com]

First of all, let me say thank you to all of you.  I have asked many
questions (some of them are dump questions), and you have kindly
helped me.  I am not going to reply every message to say thank-you
since that would be annoying for such group with such high daily
traffics.  Thank you very much.

Let's get back to topic of this message.
Here's how I have implemented it so far, and I am taking the queue of
work load items approach.
In my child thread, I will keep checking for available work load item
until a duration is reached.
#inside the child#
while endTime > time.time():
try:
 item = self.q.get(True, 3)
except Queue.Empty:  #what's wrong?  AttributeError: class
Queue has no attribute 'Empty'
 print 'cant find any work load item, so lets wait and
try again later'
 time.sleep(1) #wait and then check again
 continue
except:
 print "Unexpected error:", sys.exc_info()[0]
 raise
#do the real work with load item

In my parent thread, I will initialize X (depending on a cfg file)
child threads and keep adding load items to a shared q until the
duration is reached.
#inside the parent#
callCounter = 0
workers = [] #a list of child threads
totalWorkers = 250
endTime = time.time() + duration
for i in range(totalWorkers):
w = Worker(q, duration, i)
w.start() #worker, do your job now!
workers.append(w)

while endTime > time.time():
time.sleep(1)
q.put(getWorkloadItem()) #add workload itmes
callCounter += 1 #actually can we guarantee that the
call will be sent??
 #should we ask each child to report
the number of calls they make?

for i in range(totalWorkers):
workers[i].join()# Wait for the child threads to
finish


Overall, it seems to be working now.  Though, I still have a couple of
problems to resolve.
1. I got the following error for the codes that attempt to catch Empty
Queue exception.  What's the right way to use it?
except Queue.Empty:
AttributeError: class Queue has no attribute 'Empty'

2. What's the best way to have each child thread to report the number
of requests they send when they are done?  To add the numbers to
another queue?

3. I will need to do some logging for response time as well as some
response contents.  I have two choices, one big log file for all
threads (both child and parent), and one log file for each thread.
Given the fact that I may have to log tons of data, I think opening
and maintaining a bunch of smaller logs may be better than dealing
with a big one (it may grow very fast).  Is there any best prastice
for logging in Python?  If I change my mind and go with one big log
file (pass it to each thread), is there anything I should be aware of
for multi-thread access (writting) to the same log file?

Again, thank you.
-- 
http://mail.python.org/mailman/listinfo/python-list

Form/Template Fill-in the blanks

[allan]

I'm attempting to create a generic python script that will take an EDI
document specification and a database of transactions and be able to
generate a raw EDI file (ansi X12).

I'm looking for ideas on how best to use the strengths of Python to
implement this. I've initially tackled the EDI 812 specifications and
narrowed this down to a specific implementation by a company. This
narrowed down version is what I want to achieve for output. But I want
the script to be generic enough such that I can plug-in another EDI
specification + a database transaction set and it will output the
proper raw file accordingly.

My initial thought was to use:
1. .ini files to declare the EDI configuration
2. Use SQLAlchemy as ORB to simplify access to database objects.

INI file configuration:
* A "root" INI file indicates other INI files that define each segment
of the EDI document.
* Each segment INI defines the data elements of each segment and the
behavior of the segment (one instance or multiple-instance as in a
loop, sequence order, segment code, etc.)
* Specify data elements as either constant, system function (like
datetime), database field or object method (object being the
segment's)
* Load all the ini configuration into a "template" object. Each
segment ini maps to its own segment object.

DB using SQLAlchemy
Gather a Trading Partner data and Credit Transaction (EDI 812
remember?) into one dictionary object
Gather Credit Transaction details into another dictionary object where
it can generate the multiple instance segments

The heart of the matter is how to fuse together the template and the
data from the dictionary objects efficiently. It should be generic
enough to take another set of data dictionary and another template to
generate a completely new EDI document.

I'm stuck at this fusing together thing. Is this a good approach? Is
there an easier to implement approach?

Comments, suggestions, questions please.

Allan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why aren't OrderedDicts comparable with < etc?

[Piet van Oostrum]

> Albert van der Horst  (AvdH) wrote:

>AvdH> With regard to < and > you are right.
>AvdH> But I think there is a sensible == w.r.t. dict's.
>AvdH> It is to mean that for each key   dict1(key) == dict2(key)
>AvdH> (implying that their key set must be the same)

>AvdH> [I could have used that for one of the euler problems.
>AvdH> You have a 4 by 4 field containing a red or blue square.
>AvdH> That is naturally a mapping of (1,1) ..(4,4) tuples to one
>AvdH> of the objects `blue' `red'. After moving a square you
>AvdH> want to know whether this is a map you already have encountered.]

So what's the problem?

piet$ python3
Python 3.1 (r31:73578, Jun 27 2009, 21:49:46) 
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from collections import OrderedDict
>>> a = OrderedDict()
>>> b = OrderedDict()
>>> a[1]=2
>>> b[1]=2
>>> a[3]=4
>>> b[3]=4
>>> a==b
True
>>> b[5]=6
>>> a==b
False
>>> d1 = dict((str(i), i) for i in range (10))
>>> d2 = dict((str(i), i) for i in range (10))
>>> d1 == d2
True

-- 
Piet van Oostrum 
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: p...@vanoostrum.org
-- 
http://mail.python.org/mailman/listinfo/python-list

Distinguishing active generators from exhausted ones

[Michal Kwiatkowski]

Hi,

Is there a way to tell if a generator has been exhausted using pure
Python code? I've looked at CPython sources and it seems that
something like "active"/"exhausted" attribute on genobject is missing
from the API. For the time being I am using a simple C extension to
look at f_stacktop pointer of the generator frame, which seems to
differentiate active generators from exhausted ones. See
http://bazaar.launchpad.net/~ruby/pythoscope/support-python2.3/annotate/286/pythoscope/_util.c#L16
for complete source code.

I may be missing something obvious here. Is there a better way to tell
if a given generator object is still active or not?

Cheers,
mk
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: how can a child thread notify a parent thread its status?

[MRAB]

scriptlear...@gmail.com wrote:

First of all, let me say thank you to all of you.  I have asked many
questions (some of them are dump questions), and you have kindly
helped me.  I am not going to reply every message to say thank-you
since that would be annoying for such group with such high daily
traffics.  Thank you very much.

Let's get back to topic of this message.
Here's how I have implemented it so far, and I am taking the queue of
work load items approach.
In my child thread, I will keep checking for available work load item
until a duration is reached.
#inside the child#
while endTime > time.time():
try:
 item = self.q.get(True, 3)
except Queue.Empty:  #what's wrong?  AttributeError: class
Queue has no attribute 'Empty'
 print 'cant find any work load item, so lets wait and
try again later'
 time.sleep(1) #wait and then check again
 continue
except:
 print "Unexpected error:", sys.exc_info()[0]
 raise
#do the real work with load item

In my parent thread, I will initialize X (depending on a cfg file)
child threads and keep adding load items to a shared q until the
duration is reached.
#inside the parent#
callCounter = 0
workers = [] #a list of child threads
totalWorkers = 250
endTime = time.time() + duration
for i in range(totalWorkers):
w = Worker(q, duration, i)
w.start() #worker, do your job now!
workers.append(w)

while endTime > time.time():
time.sleep(1)
q.put(getWorkloadItem()) #add workload itmes
callCounter += 1 #actually can we guarantee that the
call will be sent??
 #should we ask each child to report
the number of calls they make?

for i in range(totalWorkers):
workers[i].join()# Wait for the child threads to
finish


Overall, it seems to be working now.  Though, I still have a couple of
problems to resolve.
1. I got the following error for the codes that attempt to catch Empty
Queue exception.  What's the right way to use it?
except Queue.Empty:
AttributeError: class Queue has no attribute 'Empty'



The exception 'Empty' belongs to the module, not the class. Try
importing as:

from Queue import Queue, Empty


2. What's the best way to have each child thread to report the number
of requests they send when they are done?  To add the numbers to
another queue?


Why not? :-)


3. I will need to do some logging for response time as well as some
response contents.  I have two choices, one big log file for all
threads (both child and parent), and one log file for each thread.
Given the fact that I may have to log tons of data, I think opening
and maintaining a bunch of smaller logs may be better than dealing
with a big one (it may grow very fast).  Is there any best prastice
for logging in Python?  If I change my mind and go with one big log
file (pass it to each thread), is there anything I should be aware of
for multi-thread access (writting) to the same log file?

Again, thank you.


If you like threads then you could put the log items into a queue and
have another thread writing them to the logfile. :-)

BTW, do you really need 250 threads? Seems like a lot.

I notice that you stop putting items into the queue when endTime is
reached and also the threads terminate when endTime is reached. If items
are put into the queue faster than they're taken out (or an item is put
in just before endTime) then there might still be unprocessed items in
the queue at endTime.
--
http://mail.python.org/mailman/listinfo/python-list

RSA cryptography between Python and Java

[Rob Knop]

I've created an RSA key in Java.  I have exported the public key by
making it into a X509EncodedKeySpec and spitting out the result of
getEncoded().

I want to use this public key to encode something in python that I will
send to Java, and then decode in Java with the corresponding private
key.

Are there any python libraries that will take a public key in this
format and do RSA encoding on it?

-- 
--Rob Knop
  E-mail:rk...@pobox.com
  Home Page: http://www.pobox.com/~rknop/
  Blog:  http://www.sonic.net/~rknop/blog/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RSA cryptography between Python and Java

[Paul Rubin]

Rob Knop  writes:
> Are there any python libraries that will take a public key in this
> format and do RSA encoding on it?

Try www.trevp.com/tlslite
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't `from pkg import mod' work after `del pkg.mod'?

[ryles]

On Jul 25, 8:57 am, Piet van Oostrum  wrote:
> > ryles  (r) wrote:
> >r> According tohttp://www.python.org/doc/essays/packages.html:
> >r> "The import statement first tests whether the item is defined in the
> >r> package; if not, it assumes it is a module and attempts to load it."
> >r> However, I've noticed that once a module is imported using the
> >r> `from pkg import mod' syntax, if its name is deleted from the
> >r> package namespace then subsequent imports with `from' will fail.
>
> This is incorrectly stated. Also on the initial import it will fail, not
> just on subsequent imports.
>
> piet$ python
> Python 2.6.2 (r262:71600, Apr 16 2009, 09:17:39)
> [GCC 4.0.1 (Apple Computer, Inc. build 5250)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.>>> 
> from pkg import _impl
>
> _impl # this is from a print statement in _impl to show that it did import
> Traceback (most recent call last):
>   File "", line 1, in 
> ImportError: cannot import name _impl
>

Well, since __init__.py is executed first, I would regard it as
causing the initial (successful) import, and yours being the
subsequent one, so that the "initial" import does not fail, the
subsequent one does. Wording aside, though, I think we are on the same
page here.

>
> According to the documentation
>  the statement
> from pkg import _impl
>
> _temp = __import__('pkg', globals(), locals(), ['_impl'], -1)
> _impl = _temp._impl
>
> This fails because _temp (the imported module) doesn't have a binding
> for _impl because you deleted it.

But supposing we hadn't deleted it, and __init__.py didn't import
_impl, there would still be no binding. Yet the importer would still
import and make the module available (presumably it "knows" to do this
since it was not already in sys.modules). The question really is that
since in our example pkg._impl is still in sys.modules, why won't the
importer add it to the pkg namespace again as it previous had? I would
imagine this is either by design, or is a case that was never
considered.

> for _impl because you deleted it. By the way, if you have a `normal'
> package that doesn't do anything with the name _impl, after the import
> the pkg module namespace will have got a binding for _impl although it is not
> in your code. It will have been put there by the import code.

Yes, this was noted further down in the original post with an example.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: len() should always return something

[Rhodri James]

On Sat, 25 Jul 2009 03:50:54 +0100, Dr. Phillip M. Feldman  
 wrote:




Here's a simple-minded example:

def dumbfunc(xs):
   for x in xs:
  print x

This function works fine if xs is a list of floats, but not if it is  
single

float.  It can be made to work as follows:

def dumbfunc(xs):
   if isinstance(xs,(int,float,complex)): xs= [xs]
   for x in xs:
  print x

Having to put such extra logic into practically every function is one of  
the

annoying things about Python.


If you persist in treating  as if it was , then
your code will always be ugly, and often buggy.  Unless we're talking
natural languages, in which case Yoda-like you will sound.

Fundamentally, your problem is your assertion that it is reasonable to
allow users to treat a single object as if it were wrapped in a list.
In Python, it is not reasonable, for the reasons that you are spending
so much time complaining about.

--
Rhodri James *-* Wildebeest Herder to the Masses
--
http://mail.python.org/mailman/listinfo/python-list

Removing newlines from string on windows (without replacing)

[Tom]

This is my first post to this mailing list, so hi :)

I have an annoying problem. While I mainly use Linux when I distribute
this program to friends and on the internet, it'll get used on Windows.
So, I tested my python program on my Windows Vista dual boot, running
the same version of python (2.6) as my Linux, and got an error at the
following code.

s = sauce.replace("\n", "")

Sauce is a string, read from a file, that I need to remove newlines
from. This code works fine in Linux, but not in Windows. rstrip("\n")
won't work for me, so anybody know how to get this working on Windows?

Thanks! :D

--print "Tom"

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Distinguishing active generators from exhausted ones

[Jason Tackaberry]

On Sat, 2009-07-25 at 11:30 -0700, Michal Kwiatkowski wrote:
> Is there a way to tell if a generator has been exhausted using pure
> Python code? I've looked at CPython sources and it seems that

Upon a cursory look, after a generator 'gen' is exhausted (meaning
gen.next() has raised StopIteration), it seems that gen.gi_frame will be
None.

Cheers,
Jason.  

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Removing newlines from string on windows (without replacing)

[Rhodri James]

On Sat, 25 Jul 2009 20:27:37 +0100, Tom  wrote:


This is my first post to this mailing list, so hi :)

I have an annoying problem. While I mainly use Linux when I distribute
this program to friends and on the internet, it'll get used on Windows.
So, I tested my python program on my Windows Vista dual boot, running
the same version of python (2.6) as my Linux, and got an error at the
following code.

s = sauce.replace("\n", "")


What error?  (I don't let Vista in the house.)


Sauce is a string, read from a file, that I need to remove newlines
from. This code works fine in Linux, but not in Windows. rstrip("\n")
won't work for me, so anybody know how to get this working on Windows?


Why won't rstrip("\n") work?  Is rstrip() OK instead, or does trailing
whitespace matter to you?

To provide a partial answer, your problem probably lies in the different
ways Windows and Linux treat ends of lines.  Under Linux, "\n" is the
line terminator.  Under Windows, it's "\r\n".  If you opened your file
in "byte mode" with open("myfile.txt", "rb"), you will be given all the
bytes in the file, including those extra "\r" characters on Windows.  If
you open your file in text mode with open("myfile.txt, "r") instead, the
line endings are converted to "\n" on both Windows and Linux.

If my crystal ball has proved faulty, giving us more details will help
get a more sensible answer.

--
Rhodri James *-* Wildebeest Herder to the Masses
--
http://mail.python.org/mailman/listinfo/python-list

Re: RSA cryptography between Python and Java

[Rob Knop]

Paul Rubin  writes:

> www.trevp.com/tlslite

Thanks, but that looks like a library for setting up a secure connection
between two ends.  What I need is the ability to encrypt with a public
key in Python, where that public key was generated in Java as described,
and where the cipertext can later be decrypted with the corresponding
secret key in Java.

-- 
--Rob Knop
  E-mail:rk...@pobox.com
  Home Page: http://www.pobox.com/~rknop/
  Blog:  http://www.sonic.net/~rknop/blog/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RSA cryptography between Python and Java

[Paul Rubin]

Rob Knop  writes:
> > www.trevp.com/tlslite
> 
> Thanks, but that looks like a library for setting up a secure connection
> between two ends.  What I need is the ability to encrypt with a public
> key in Python, where that public key was generated in Java as described,
> and where the cipertext can later be decrypted with the corresponding
> secret key in Java.

Yes, I think that library has the function that you want.  It's just
X509 DER encoding.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Distinguishing active generators from exhausted ones

[Michal Kwiatkowski]

On Jul 25, 10:00 pm, Jason Tackaberry  wrote:
> On Sat, 2009-07-25 at 11:30 -0700, Michal Kwiatkowski wrote:
> > Is there a way to tell if a generator has been exhausted using pure
> > Python code? I've looked at CPython sources and it seems that
>
> Upon a cursory look, after a generator 'gen' is exhausted (meaning
> gen.next() has raised StopIteration), it seems that gen.gi_frame will be
> None.

Only in Python 2.5 or higher though. I need to support Python 2.3 and
2.4 as well, sorry for not making that clear in the original post.

Cheers,
mk
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: available formats and params for Image.save()

[News123]

Thanks a lot Gabriel,

Your answer is perfect.

I was only looking in part II of the PIL manual and overlooked the
Appendix :-(.

I also apreciate
Image.ID , Image.SAVE , Image.OPEN

I saw them when typing
dir Image
but I never had the idea of calling Image.init() first

bye

N




Gabriel Genellina wrote:
> En Thu, 23 Jul 2009 06:56:45 -0300, News123  escribió:
> 
>> Somehow I have difficulties reading the documentation for PIL (Image)
>>
>> Is there an easy way to know which formats are supported and what their
>> names are?
> 
> py> import PIL
> py> from PIL import Image
> py> Image.ID
> []
> py> Image.init()
> py> Image.ID
> ['PNG', 'ARG', 'BMP', 'BUFR', 'CUR', 'PCX', 'DCX', 'EPS', 'FITS', 'FLI',
> 'FPX',
> 'GBR', 'GIF', 'GRIB', 'HDF5', 'ICNS', 'ICO', 'IM', 'IMT', 'IPTC',
> 'JPEG', 'MCIDA
> S', 'TIFF', 'MIC', 'MPEG', 'MSP', 'PCD', 'PIXAR', 'PPM', 'PSD', 'SGI',
> 'SPIDER',
>  'SUN', 'TGA', 'WBMP', 'WMF', 'XBM', 'XPM', 'XVTHUMB']
> py> Image.OPEN.keys()
> ['PCX', 'ICNS', 'HDF5', 'SUN', 'MIC', 'EPS', 'MSP', 'FLI', 'FITS',
> 'GBR', 'WBMP'
> , 'PCD', 'PIXAR', 'BUFR', 'PPM', 'WMF', 'SGI', 'BMP', 'TGA', 'DCX',
> 'ICO', 'CUR'
> , 'XPM', 'TIFF', 'JPEG', 'SPIDER', 'GIF', 'GRIB', 'IM', 'IMT', 'IPTC',
> 'FPX', 'X
> BM', 'MPEG', 'PSD', 'ARG', 'XVTHUMB', 'PNG', 'MCIDAS']
> py> Image.SAVE.keys()
> ['XBM', 'PCX', 'SPIDER', 'HDF5', 'TIFF', 'BUFR', 'EPS', 'JPEG', 'MSP',
> 'GRIB', '
> GIF', 'BMP', 'IM', 'PPM', 'PDF', 'FITS', 'PALM', 'WBMP', 'WMF', 'PNG']
> 
>> Is there an easy way to know which parameters are supported by
>> Image.save(). How can I list them where are they documented?
> 
> That depends on the format being used. The PIL handbook lists the
> standard formats used and their parameters:
> http://www.pythonware.com/library/pil/handbook/index.htm
> 
>> I'm at a complete loss at finding out what parameters the save function
>> accepts for saving a JPG file or a PNG file
> 
> http://www.pythonware.com/library/pil/handbook/format-jpeg.htm
> 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: len() should always return something

[Robert Kern]

On 2009-07-24 21:50, Dr. Phillip M. Feldman wrote:

Here's a simple-minded example:

def dumbfunc(xs):
for x in xs:
   print x

This function works fine if xs is a list of floats, but not if it is single
float.  It can be made to work as follows:

def dumbfunc(xs):
if isinstance(xs,(int,float,complex)): xs= [xs]
for x in xs:
   print x

Having to put such extra logic into practically every function is one of the
annoying things about Python.


I have spent the last ten years writing scientific code in Python (i.e. that 
which otherwise might be written in Matlab), and I can guarantee you that you do 
not need to put such extra logic in practically every function. Even when 
naively translating code from Matlab, it's not often necessary.


By the way, are you familiar with numpy? If you are converting code from Matlab, 
you will almost certainly need it. We have a number of functions that make these 
kinds of operations easy when they are in fact necessary. For example, we have 
isscalar() and atleast_1d().


  def dumbfunc(xs):
xs = numpy.atleast_1d(xs)
for x in xs:
  print x

http://numpy.scipy.org/

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list

Re: len() should always return something

[Robert Kern]

On 2009-07-25 02:55, Diez B. Roggisch wrote:

Dr. Phillip M. Feldman schrieb:

Here's a simple-minded example:

def dumbfunc(xs):
for x in xs:
print x

This function works fine if xs is a list of floats, but not if it is
single
float. It can be made to work as follows:

def dumbfunc(xs):
if isinstance(xs,(int,float,complex)): xs= [xs]
for x in xs:
print x

Having to put such extra logic into practically every function is one
of the
annoying things about Python.


And where comes "len(xs)" into play here? What you want is iteration
over scalars.


He explained in another post that iteration is another feature along the same 
lines that he would want for scalars.


--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list

Failed Regression Test: What socket.gethostname() is supposed to return?

[Lie Ryan]

In my laptop, socket.gethostname() returned my username, and causing one
of python's "make test" regression test to error (test_socket):

==
ERROR: testSockName (test.test_socket.GeneralModuleTests)
--
Traceback (most recent call last):
  File
"/var/tmp/portage/dev-lang/python-2.5.4-r3/work/Python-2.5.4/Lib/test/test_socket.py",
line 456, in testSockName
my_ip_addr = socket.gethostbyname(socket.gethostname())
gaierror: (-2, 'Name or service not known')

--

since on my system socket.gethostname() returns 'lieryan', and since
socket.gethostbyname('lieryan') does not resolve to anything; the test
becomes an error.

My system is Gentoo, but I think this also happened on Ubuntu (still on
this laptop). The trunk failed in similar manner.

Do I have a misconfigured system or is the test faulty?

For convenience, the relevant test code (taken from trunk):

===
def testSockName(self):
# Testing getsockname()
port = self._get_unused_port()
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind(("0.0.0.0", port))
name = sock.getsockname()
# XXX(nnorwitz): http://tinyurl.com/os5jz seems to indicate
# it reasonable to get the host's addr in addition to 0.0.0.0.
# At least for eCos.  This is required for the S/390 to pass.
my_ip_addr = socket.gethostbyname(socket.gethostname())
self.assertTrue(name[0] in ("0.0.0.0", my_ip_addr), '%s invalid' %
name[0])
self.assertEqual(name[1], port)
===


lier...@lieryan ~/Desktop/pythontrunk/trunk $ ./python -m test.regrtest
test_socket
Could not find '/home/lieryan/Desktop/pythontrunk/trunk/Lib/test' in
sys.path to remove it
test_socket
test test_socket failed -- Traceback (most recent call last):
  File
"/home/lieryan/Desktop/pythontrunk/trunk/Lib/test/test_socket.py", line
493, in testSockName
my_ip_addr = socket.gethostbyname(socket.gethostname())
gaierror: [Errno -2] Name or service not known

1 test failed:
test_socket

I tracked the code for socket.gethostname() and socket.gethostbyname()
and found that they are simply a wrapper for gethostname() from #import
 (http://linux.die.net/man/2/gethostname) and gethostbyname()
from #import  (http://linux.die.net/man/3/gethostbyname). A
simple test in C found that the C's equivalent to
gethostbyname(gethostname()) returns a null pointer (used to indicate
error, per documentation).

So, the question is: what is socket.gethostname() is supposed to return
that will be a valid argument for socket.gethostbyname()?

PS: I found an MSDN article by Microsoft stating that
gethostbyname(gethostname) is guaranteed to always succeed
(http://msdn.microsoft.com/en-us/library/ms738527(VS.85).aspx); is this
guarantee also true in linux?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: len() should always return something

[Marcus Wanner]

On 7/25/2009 10:08 AM, Piet van Oostrum wrote:

Steven D'Aprano  (SD) wrote:


SD> Ambiguity essentially boils down to being unable to reasonably predict 
SD> the expectation of the coder. I say "reasonably", because if you allow 
SD> unreasonable situations, everything is "ambiguous":


That's for me the reason that len(42) is ambiguous. The OP apparently
had 1 as expectation, whereas my first thought was the minimal number of
bits to represent the number and 7.5 million came later :=). The number
of bits I certainly find reasonable, and I would find the number of
decimal digits equally reasonable. More so than 1, actually. 1 as the
length of an int doesn't give any information.
Well, for my two cents, I will say that the number of binary bits or 
decimal digits is certainly the most sensible return value, and that the 
former is the most useful, because the latter can be got with 
len(str(42)). However, the former can also be (/slightly/ less)easily 
got with len(bin(42))-2...


I also think that "Explicit is better than implicit." says that there 
should be no return value in this case, as any return value would be 
implicit.


Marcus
--
http://mail.python.org/mailman/listinfo/python-list

Re: Distinguishing active generators from exhausted ones

[Ben Finney]

Michal Kwiatkowski  writes:

> I may be missing something obvious here. Is there a better way to tell
> if a given generator object is still active or not?

foo = the_generator_object
try:
do_interesting_thing_that_needs(foo.next())
except StopIteration:
generator_is_exhausted()

In other words, don't LBYL, because it's EAFP. Whatever you need to do
that requires the next item from the generator, do that; you'll get a
specific exception if the generator is exhausted.

-- 
 \  “Courteous and efficient self-service.” —café, southern France |
  `\   |
_o__)  |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: len() should always return something

[Erik Max Francis]

Steven D'Aprano wrote:
But it's not "practically every function". It's hardly any function at 
all -- in my code, I don't think I've ever wanted this behavior. I would 
consider it an error for function(42) and function([42]) to behave the 
same way. One is a scalar, and the other is a vector -- they're different 
things, it's poor programming practice to treat them identically.


(If Matlab does this, so much the worse for Matlab, in my opinion.)


There's actually good reason to do this in heavily matrix-oriented 
specialized languages; there are numerous applications where scalars and 
1x1 matrices are mathematically equivalent.


--
Erik Max Francis && m...@alcyone.com && http://www.alcyone.com/max/
 San Jose, CA, USA && 37 18 N 121 57 W && AIM/Y!M/Skype erikmaxfrancis
  Gods are born and die, but the atom endures.
   -- Alexander Chase
--
http://mail.python.org/mailman/listinfo/python-list

Re: len() should always return something

[Chris Rebert]

On Sat, Jul 25, 2009 at 4:21 PM, Erik Max Francis wrote:
> Steven D'Aprano wrote:
>>
>> But it's not "practically every function". It's hardly any function at all
>> -- in my code, I don't think I've ever wanted this behavior. I would
>> consider it an error for function(42) and function([42]) to behave the same
>> way. One is a scalar, and the other is a vector -- they're different things,
>> it's poor programming practice to treat them identically.
>>
>> (If Matlab does this, so much the worse for Matlab, in my opinion.)
>
> There's actually good reason to do this in heavily matrix-oriented
> specialized languages; there are numerous applications where scalars and 1x1
> matrices are mathematically equivalent.

The pertinent issue here being that Python, as a language, is neither
matrix-oriented nor special-purpose. :)

Cheers,
Chris
-- 
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: strange python scripting error

[Dave Angel]

Dennis Lee Bieber wrote:

On Sat, 25 Jul 2009 07:18:58 -0400, Dave Angel 
declaimed the following in gmane.comp.python.general:

  
Another thing I'd point out is that some ftp programs will do this 
conversion as the file is being sent between a local DOS machine and a 
Unix machine on the internet.




"some ftp programs"?

Line end conversion is the whole reason, to my knowledge, for text
mode transfer, vs binary mode transfer.



  
Perhaps I should have said "most ftp programs," or even "all ftp 
programs," but I don't feel qualified to make such a generalization.  In 
any case, I don't currently use such automatic conversion when I'm doing 
transfers because I like to be able to see exactly what's there, and I 
don't have another way to directly examine the remote end.


The FTP I use is an add-on for Firefox (FireFTP), and although it has 
text and "auto" modes, it's buried in the Tools->options menu, and the 
main panel just does a transfer using whichever mode was selected.  
Rather than risking corrupting the (mostly binary) files, I choose to 
use binary mode all the time.


DaveA

--
http://mail.python.org/mailman/listinfo/python-list

Re: how can a child thread notify a parent thread its status?

[scriptlear...@gmail.com]

I decided to go with one big log file, which will be shared by all
threads (child and parent).  A log message Queue is used to store all
log entries, and a customized logger thread will get log entries from
the Queue.

#from the logger thread#
def run(self):
while self.flag == 1: #if the flag is set to 0, the logger
thread should exit
try:
 entry = self.q.get()
except Empty:
 self.logger.debug('cant find any log entry')
 continue
except:
 self.logger.error("Unexpected error:", sys.exc_info()
[0])
 raise
#do whatever that should be done
self.logger.info("logger thread done") #should see this
message in log file as well
def off(self):
self.logger.info('turning off flag')
self.flag = 0


#in parent thread#
logItemQ.put('We are done, lets stop the logger now.')
time.sleep(1) #it seems that the logger thread cannot exit if
I put a sleep here
myLog.off() #off is called successfully
myLog.join()


I put an off method to turn off a flag so the logger thread knows it
should exit.  However, the last log message (the one 'We are done,
lets stop the logger now.') won't be logged if I call myLog.off() and
myLog.join() immediately.  So, I put a time.sleep(1) to make sure the
logger thread has enough time to finish it's job.  Unfortunately, now
the logger thread simply won't exit, and I don't see the message
'logger thread done'.  I can't figure out at which point it hangs,
since I don't any new log entry but the thread simply won't exit.
Am I taking a right approach by using a flag?  Should I lock the flag?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: how can a child thread notify a parent thread its status?

[MRAB]

scriptlear...@gmail.com wrote:

I decided to go with one big log file, which will be shared by all
threads (child and parent).  A log message Queue is used to store all
log entries, and a customized logger thread will get log entries from
the Queue.

#from the logger thread#
def run(self):
while self.flag == 1: #if the flag is set to 0, the logger
thread should exit
try:
 entry = self.q.get()
except Empty:
 self.logger.debug('cant find any log entry')
 continue
except:
 self.logger.error("Unexpected error:", sys.exc_info()
[0])
 raise
#do whatever that should be done
self.logger.info("logger thread done") #should see this
message in log file as well
def off(self):
self.logger.info('turning off flag')
self.flag = 0


#in parent thread#
logItemQ.put('We are done, lets stop the logger now.')
time.sleep(1) #it seems that the logger thread cannot exit if
I put a sleep here
myLog.off() #off is called successfully
myLog.join()


I put an off method to turn off a flag so the logger thread knows it
should exit.  However, the last log message (the one 'We are done,
lets stop the logger now.') won't be logged if I call myLog.off() and
myLog.join() immediately.  So, I put a time.sleep(1) to make sure the
logger thread has enough time to finish it's job.  Unfortunately, now
the logger thread simply won't exit, and I don't see the message
'logger thread done'.  I can't figure out at which point it hangs,
since I don't any new log entry but the thread simply won't exit.
Am I taking a right approach by using a flag?  Should I lock the flag?


self.q.get() will block if the queue is empty, so the Empty exception is
never raised.

What's happening is that the parent thread puts the final message into
the queue, sleeps, and then clears the flag; meanwhile, the logging
thread gets the message, writes it out, checks the flag, which is still
set, and then tries to get the next message. The queue is empty, so the
.get() blocks.

The simplest solution is not to use a flag, but the sentinel trick. The
parent thread can put, say, None into the queue after the last message;
when the logging thread gets None, it knows it should terminate.

--
http://mail.python.org/mailman/listinfo/python-list

Re: len() should always return something

[Erik Max Francis]

Chris Rebert wrote:

On Sat, Jul 25, 2009 at 4:21 PM, Erik Max Francis wrote:

Steven D'Aprano wrote:

But it's not "practically every function". It's hardly any function at all
-- in my code, I don't think I've ever wanted this behavior. I would
consider it an error for function(42) and function([42]) to behave the same
way. One is a scalar, and the other is a vector -- they're different things,
it's poor programming practice to treat them identically.

(If Matlab does this, so much the worse for Matlab, in my opinion.)

There's actually good reason to do this in heavily matrix-oriented
specialized languages; there are numerous applications where scalars and 1x1
matrices are mathematically equivalent.


The pertinent issue here being that Python, as a language, is neither
matrix-oriented nor special-purpose. :)


Yes.  And I was responding to the comment that such a feature of a 
language would a priori be poor design.  It _isn't_ poor design for 
special purpose languages.  Python isn't one of them, but Matlab _is_.


--
Erik Max Francis && m...@alcyone.com && http://www.alcyone.com/max/
 San Jose, CA, USA && 37 18 N 121 57 W && AIM/Y!M/Skype erikmaxfrancis
  More fodder for the new lost generation
   -- Nik Kershaw
--
http://mail.python.org/mailman/listinfo/python-list

Re: len() should always return something

[Chris Rebert]

On Sat, Jul 25, 2009 at 6:47 PM, Erik Max Francis wrote:
> Chris Rebert wrote:
>>
>> On Sat, Jul 25, 2009 at 4:21 PM, Erik Max Francis wrote:
>>>
>>> Steven D'Aprano wrote:

 But it's not "practically every function". It's hardly any function at
 all
 -- in my code, I don't think I've ever wanted this behavior. I would
 consider it an error for function(42) and function([42]) to behave the
 same
 way. One is a scalar, and the other is a vector -- they're different
 things,
 it's poor programming practice to treat them identically.

 (If Matlab does this, so much the worse for Matlab, in my opinion.)
>>>
>>> There's actually good reason to do this in heavily matrix-oriented
>>> specialized languages; there are numerous applications where scalars and
>>> 1x1
>>> matrices are mathematically equivalent.
>>
>> The pertinent issue here being that Python, as a language, is neither
>> matrix-oriented nor special-purpose. :)
>
> Yes.  And I was responding to the comment that such a feature of a language
> would a priori be poor design.  It _isn't_ poor design for special purpose
> languages.  Python isn't one of them, but Matlab _is_.

I was agreeing with your point actually. That was what I was trying to
convey in my post. Apparently I wasn't as successful in that regard as
I'd hoped.

- Chris
-- 
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: uniicode and executing a process with subprocess.call, or os.system

[Martin v. Löwis]

> I am very confused about unicode. Can someone point me in the right
> direction?

Try Python 3.1. This should accept Unicode strings directly for sp.call,
so you wouldn't need to encode first.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: If Scheme is so good why MIT drops it?

[Raffael Cavallaro]

On 2009-07-25 00:55:26 -0400, Carl Banks  said:


But please don't put it on the same level as PHP.  Their situations
have almost nothing in common.


Their situations have much in common; Python attracted programmers away 
from (for example) C++, becuse python is easier to master; Then php 
came along and attracted programmers away from (for example) python, 
because php is easier to master.


This doesn't mean they're on the same level - in fact, if you read 
carefully you'll see my original post said as much: python attracted 
average programmers; php attracted mediocre programmers and even some 
non-programmers, which means that php is clearly a lesser language than 
python.

--
Raffael Cavallaro

--
http://mail.python.org/mailman/listinfo/python-list

multiprocessing and __main__

[is un]

Hi,

Trying the multiprocessing module for the first time, I spent quite a
bit on making this code run:

from multiprocessing import Process
import time

def my_process():
i = 0
while 1:
print i
i += 1
time.sleep(0.5)


p = Process(target=my_process, args=())
p.start()

print 'Process started'

The result was not what I expected, it seems like the process restarts
all the time, and the message 'Process started' keeps getting
printed...

Going back to the documentation, I realized that the doc was really
serious about the line:
if __name__ == '__main__' .. which I always ignore, and by changing
the code to

if __name__ == '__main__':
p = Process(target=my_process, args=())
p.start()

print 'Process started'

the problem was solved.
So my question is what actually happens when I call p.start()? Is the
entry file reloaded under a different module name?

Thanks
iu2
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: MySQLdb for Python 3.1 getting close?

[John Nagle]

(This is actually a repost; outgoing netnews server was down for a while.)
John Nagle wrote:

  Is MySQLdb available for Python 3.1 yet?

  http://sourceforge.net/projects/mysql-python/

says the last supported Python version is 2.5.  Any progress in sight?

John Nagle   

--
http://mail.python.org/mailman/listinfo/python-list

Re: multiprocessing and __main__

[is un]

Me again, fix a type in my question:
 So my question is what actually happens when I call p.start()? Is the
 *entire* file reloaded under a different module name?
>
> Thanks
> iu2

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: non-owning references?

[John Nagle]

Peter Otten wrote:

Utpal Sarkar wrote:


Is there a way I can tell a variable that the object it is pointing
too is not owned by it, in the sense that if it is the only reference
to the object it can be garbage collected?


http://docs.python.org/library/weakref.html


   Yes.  Weak references can be quite useful.  I have a version of
BeautifulSoup in which all the "upward" and "backwards" links are
weak references, but the "downward" and "forwards" links are strong
references.  This breaks the loops in the data structure, and thus
trees and subtrees will go away when no longer required, without
requiring a GC cycle.

John Nagle
--
http://mail.python.org/mailman/listinfo/python-list

Re: code debugging

[Chris Rebert]

On Sat, Jul 25, 2009 at 11:23 PM, golu wrote:
> here is a code which crawls links sent to it. theres some problem with
> the retrieve_url function ,plz help me out in debugging the fuction
> retrive_url. This function retrives pages and saves them in file

Please specify exactly what the problem is that you are experiencing.
If you are getting an error, please provide the error message and full
traceback.

Cheers,
Chris
-- 
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

code debugging

[golu]

here is a code which crawls links sent to it. theres some problem with
the retrieve_url function ,plz help me out in debugging the fuction
retrive_url. This function retrives pages and saves them in file
#TODO:Visited dict grows in size it needs to be handled smartly
#Moreover server program needs to be in sync with the client eg.
Myrobot
#Take care of tag - 'if modified since',repeated links,hash links
#This is the client side of the distributed crawling framework
#It gets the list of urls to be crawled
#Then crawls the urls and stores the pages in a temporary archive
#which is then transferred to the server or grey_matter
import httplib
import os
import sys
import urlparse
import urllib2
import urllib
import zipfile
import threading

from socket import *
PAGE_DIR="C:/users/jayesh/
pages/"  # directory where the
web pages are stored temporarily
 
# before transfer to the grey_matter
visited=
{} # a
dict to remember visited urls
ROBOT_COUNT=4


def fget():
""" This function retrieves the zipped file
 containing the list of urls from the grey_matter and
 saves them in a local file 'list.txt'. """

httplib.HTTPConnection.debuglevel=1
request=urllib2.Request('http://192.168.153.57/list.zip')
#Requesting the zipped file
request.add_header('Accept-encoding','gzip')   #containing
the list of urls
opener=urllib2.build_opener()
flag=1
s='Waiting for server'
while flag==1:
 try:
  op=opener.open(request)
  flag=0
 except:
 s=s+'*'
 print s
f=open('list.zip',"wb")
f.write(op.read())
f.close()
z=zipfile.ZipFile('list.zip')
p=z.namelist()
g=open('list.txt',"wb")
g.write(z.read(p[0]))
g.close()
print 'got zipped file'

def compress():
""" This function compresses the crawled pages and stores them in
a single compressed file ready to be sent to the
grey_matter."""

zfile=zipfile.ZipFile('C:/xampp/htdocs/pages.zip',mode='w')
for fil in os.listdir(PAGE_DIR):
full=os.path.join(PAGE_DIR,fil)
zfile.write(full,fil)
os.remove(full)
os.rmdir(PAGE_DIR) #Removing the directory after
transfer to grey_matter


x=0
class robot(threading.Thread):
""" The main robot class which does the crawling of listed
urls it recieves from the grey matter. It uses 3 threads which
crawl the listed urls synchronously."""

def __init__(self,urllist,urllistlock,dblock):
threading.Thread.__init__(self)
self.urllist=urllist
self.urllistlock=urllistlock
self.dblock=dblock

def popurl(self):
""" This method pops out urls from the urls file one by one
and sends them for retrieval."""

self.urllistlock.acquire(1)
if(len(self.urllist)<1):
Nexturl=None
else:
Nexturl=self.urllist[0]
if Nexturl[-1]=='\n':Nexturl=Nexturl[:-1]
del self.urllist[0]
self.urllistlock.release()
return Nexturl

def retrieve_url(self,url):
""" The main method of the robot class and is called
run method to retrieve the given urls from the web."""
global x
if url is not None:

 try:
if visited.has_key(url): return
pieces=urlparse.urlparse(url)
filepath=pieces[2]
if filepath != '':
 filepath=filepath[1:]
 filename=filepath.split("/")[-1]
else:
  filename=x+'.htm'
  x+=1

path=os.path.join(PAGE_DIR,filename)
url=urlparse.urlunparse(pieces)
p=url.rfind('#')   #temporary
if p!=-1:
url=url[:p]

visited[url]=1
m=urllib2.urlopen(url)

fopen=open(path,'wb')

fopen.seek(0)
fopen.write(url+'|')

fopen.write(m.read())
fopen.close()
print url ,'retrieved'

 except IOError:
print url
print "ERROR:OOPS! THE URL CAN'T BE RETRIEVED"

return

def run(self):
while(1):
url=self.popurl()
if url is None:
break
try:
 self.retrieve_url(url)
except:sys.exit()

if __name__=='__main__':

  s=socket(AF_INET,SOCK_STREAM)
  s.bind(('',444))
  s.listen(5)
  q,v=s.accept()
  count=1
  print 'Connecting...'
  while 1:
print 'Phase: %s' %(count)
message=q.recv(3)

if(message!='yes'):continue
print 'Connected'
count=count+1
fget() # Calling the fget method to get the url list
from
   # grey_matter(server).
try:
 os.mkdir(PAGE_DIR)
except: print 'Cant make dir'
try:
 f=open('list.txt','r')
 urllist=f.readlines()
 f.close()
except:
print 'Error opening ur

Re: code debugging

[golu]

On Jul 26, 11:28 am, Chris Rebert  wrote:
> On Sat, Jul 25, 2009 at 11:23 PM, golu wrote:
> > here is a code which crawls links sent to it. theres some problem with
> > the retrieve_url function ,plz help me out in debugging the fuction
> > retrive_url. This function retrives pages and saves them in file
>
> Please specify exactly what the problem is that you are experiencing.
> If you are getting an error, please provide the error message and full
> traceback.
>
> Cheers,
> Chris
> --http://blog.rebertia.com

i want to save pages in a directory and i m using the urls to get
filenames. The program gets stuck in the saving step.can u suggest me
a way to save a page e.g google.com as a file google.html
-- 
http://mail.python.org/mailman/listinfo/python-list