Re: PyPI editing

[Chris Rebert]

On Wed, Mar 4, 2009 at 11:42 PM, andrew cooke  wrote:
>
> Not sure where to ask this, but how do I edit my PyPI page?
>
> http://pypi.python.org/pypi/LEPL/2.0 doesn't have any text compared to
> http://pypi.python.org/pypi/pypp/0.0.2 (selected at random).  How do I the
> "Benefits", "Drawbacks" etc?
>
> I have clicked around the admin interface, but I only see what seem to be
> very specific fields in the "edit" section.  If one of those can take
> HTML, which is it?

I would venture a guess that the main description field accepts
reStructuredText formatting.

Cheers,
Chris

-- 
I have a blog:
http://blog.rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3D CAD -- need collaborators, or just brave souls :)

[jelle feringa]

Hi Josh,

> http://www.pythonocc.org/ However, I'm
> not entirely clear on the license for this so that might be an issue.
 
We're using a French license for the moment, but will move to something more
standard soon. PythonOCC ( the current SVN version ) wraps 85% of the 
OpenCASCADE kernel. Consider that complete, since there are a bunch of 
modules are obsolete ( WOK, drawing ).

(Binaries are supplied for win32, linux & osx.)

We're starting to work on a high level API, so this is a wonderful moment to
jump on.


-jelle

--
http://mail.python.org/mailman/listinfo/python-list

Can't override class |__new__

[jelle feringa]

Hi,

I'm working with a C++ module ( CGAL, comp.geom. with exact arithmic )
and am having troubles finding a way to override how the modules returns
objects. What I'm trying to do is to extend the Facet class, but when I try
to use my version of the class, the parent class is still being returned

import CGAL

CGAL.Facet = OtherFacet
CGAL.Polyhedron.Facet = OtherFacet

p = CGAL.Polyhedron_3()
p.make_triangle()

for f in p.facets:
print f


# Here I was expecting a OtherFacet object

Is there a way of getting around this, or is this behaviour hardcoded in the
C++ module?
Can I perhaps alter it using  the new module or __new__ method?

Many thanks in advance,

-jelle

--
http://mail.python.org/mailman/listinfo/python-list

Re: PyPI editing

[andrew cooke]

Apparently not (this will probably change, but see
http://pypi.python.org/pypi/LEPL/2.0.1 which is currently displaying
restructured text literally)

Thanks,
Andrew

Chris Rebert wrote:
> On Wed, Mar 4, 2009 at 11:42 PM, andrew cooke  wrote:
>>
>> Not sure where to ask this, but how do I edit my PyPI page?
>>
>> http://pypi.python.org/pypi/LEPL/2.0 doesn't have any text compared to
>> http://pypi.python.org/pypi/pypp/0.0.2 (selected at random).  How do I
>> the
>> "Benefits", "Drawbacks" etc?
>>
>> I have clicked around the admin interface, but I only see what seem to
>> be
>> very specific fields in the "edit" section.  If one of those can take
>> HTML, which is it?
>
> I would venture a guess that the main description field accepts
> reStructuredText formatting.
>
> Cheers,
> Chris
>
> --
> I have a blog:
> http://blog.rebertia.com
>
>


--
http://mail.python.org/mailman/listinfo/python-list

Re: qt, gtk, wx for py3 ?

[Peter Billam]

> On 4 Mar, 00:40, Peter Billam  wrote:
>>  http://wiki.wxwidgets.org/WxWidgets_Compared_To_Other_Toolkits
>> seemed surprisingly even-handed.

On 2009-03-04, Paul Boddie  wrote:
> I don't have a horse in this race, but apart from needing to
> update that page based on recent developments (at least for Qt: a
> licensing change which, being someone in favour of Free Software,
Likewise...

> I'm not particularly enthusiastic about),
On http://www.riverbankcomputing.co.uk/software/pyqt/intro ...
  "PyQt does not include a copy of Qt. You must
   obtain a correctly licensed copy of Qt yourself."
Does that refer to what you're refering to ?

>> On 2009-03-03, Mike Driscoll  wrote:
>> > It should be noted that the port for 3.0 hasn't started yet for
>> > wxPython and I'm not seeing anything about a port for PyQt either
>> > on their website.

I mailed riverbankcomputing about PyQt for Py3; Phil Thompson replied
> The next release of PyQt is planned for the end of this month.
> It *might* make it to that release.

which, *if* it happens, is not so far away.
But that still leaves Gtk and Wx...

> Take a look at this thread:
> http://www.mail-archive.com/p...@riverbankcomputing.com/msg15072.html
Good thread...
> http://www.gtk.org/features.html
Thank you...
> http://www.qtsoftware.com/products/platform/qt-for-embedded-linux/
Likewise...

> I don't think it's sensible to hang around waiting for Python 3
> library availability. You'd be better served using Python 2 straight
> away and then considering moving to Python 3 when such libraries
> finally become available.
> Paul

I might end up doing that, to get the feel of Qt v. Gtk v. Wx,
but I'd be unproductive in that mode because my app needs a module
I've just written in Py3, having bought Mark Summerfield's book.

Thanks,  Regards,  Peter

-- 
Peter Billam   www.pjb.com.auwww.pjb.com.au/comp/contact.html
--
http://mail.python.org/mailman/listinfo/python-list

Re: 2to3 Help?

[John Machin]

On Mar 5, 6:47 pm, jjh  wrote:
> On Jan 14, 10:01 pm, marco.m.peter...@gmail.com wrote:
[snipped]

> Sorry to bring this back up.

The correct thing to do is to start a NEW post. Marco's problem is
nothing to do with yours.




> I'm a newbie trying to run 2to3 on
> Windows. I ran the following:
>
> C:\Python30\python.exe C:\Python30\Tools\Scripts\2to3.py 
> c:\python\src\testscript.py

The .exe should not be necessary ... if you have put a python.bat or a
python.com or similar in that directory, lose them soon.

>
> and I got the following:
>
> C:\Users\user>C:\Python30\python.exe C:\Python30\Tools\Scripts\2to3.py
> c:\python\src\testscript.py
> RefactoringTool: Skipping implicit fixer: buffer
> RefactoringTool: Skipping implicit fixer: idioms
> RefactoringTool: Skipping implicit fixer: set_literal
> RefactoringTool: Skipping implicit fixer: ws_comma
> --- c:\python\src\testscript.py (original)
> +++ c:\python\src\testscript.py (refactored)
> @@ -1,3 +1,3 @@
>  #! /usr/bin/python
>
> -print "Hello World";
> +print("Hello World");
> RefactoringTool: Files that need to be modified:
> RefactoringTool: c:\python\src\testscript.py
>
> what am I suppose to do after that? I thought it would re-write my
> testscript.py but it didn't. Thanks so much for the help.

What made you think so? Thinking is a very good idea, but when you are
a new user of a piece of software it's an even better idea to read the
manual (http://docs.python.org/3.0/library/2to3.html) and check that
it follows the usual command-line help convention (c:\python30\python
c:\python30\tools\scripts\2to3.py --help) and that the fancy Windows
help for Python is working ((Windows XP) click on Start>All
Programs>Python 3.0>Python Manuals, type 2to3 then Enter) just in case
you are ever disconnected from the Internet and/or want faster
response.

HTH,
John
--
http://mail.python.org/mailman/listinfo/python-list

Re: Peculiar swap behavior

[Aaron Brady]

On Feb 23, 12:43 pm, Tim Chase  wrote:
> I stumbled across this oddity and was hoping folks on the list
> might be able to provide a little understanding:
>
> # swap scalars
>  >>> x,y = 1,2
>  >>> x,y = y,x
>  >>> x,y
> (2, 1)
>
> # swap lists
>  >>> a,b = [1,2,3],[4,5,6]
>  >>> a,b = b,a
>  >>> a,b
> ([4, 5, 6], [1, 2, 3])
>
> # swap list contents...not so much...
>  >>> m,n = [1,2,3],[4,5,6]
>  >>> m[:],n[:] = n,m
>  >>> m,n
> ([4, 5, 6], [4, 5, 6])
snip

This may help:

>>> a= [1,2,3]
>>> id(a)
12170584
>>> id(a[:])
12235520

However, I'm not thinking in English at the moment.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Can't override class |__new__

[Aaron Brady]

On Mar 5, 2:27 am, jelle feringa  wrote:
> Hi,
>
> I'm working with a C++ module ( CGAL, comp.geom. with exact arithmic )
> and am having troubles finding a way to override how the modules returns
> objects. What I'm trying to do is to extend the Facet class, but when I try
> to use my version of the class, the parent class is still being returned
>
> import CGAL
>
> CGAL.Facet = OtherFacet
> CGAL.Polyhedron.Facet = OtherFacet
>
> p = CGAL.Polyhedron_3()
> p.make_triangle()
>
> for f in p.facets:
>     print f
>
> 
> # Here I was expecting a OtherFacet object
>
> Is there a way of getting around this, or is this behaviour hardcoded in the
> C++ module?
> Can I perhaps alter it using  the new module or __new__ method?
>
> Many thanks in advance,
>
> -jelle

Hello.  I don't have your module, but it sounds like the C is doing
something like this (pseudocode):

PyObject *make_triangle(...) {
...
new_ob= PyObject_New( &CGAL_Polyhedron_Facet_Type );
...
}

Regardless of CGAL's dictionary, it instantiates a Facet.  What you
want it to do is (pseudocode):

PyObject *make_triangle(...) {
...
/* get the current 'Facet' member */
class_ob= PyObject_GetAttr( CGAL, "Facet" );
/* instantiate it */
new_ob= PyObject_New( class_ob );
...
}

Depending on the details, you may need only to cut-and-paste your own
'make_triangle' function, and just replace the line I showed.

Do you have a link to the source for CGAL?
--
http://mail.python.org/mailman/listinfo/python-list

Does profiling remove some optimisations?

[andrew cooke]

I have a core loop that is critical to performance.  The code is at
http://www.acooke.org/lepl/api/lepl.parser-pysrc.html#trampoline

If I write a separate "optimised" version of that function for when
"monitor" is empty, with all the "if monitor" tests removed, the profiler
(cProfile) indicates a 10% reduction in time spent in the loop.

But if I run the same code 100 times under timeit, without profiling, I
see no difference in total time.  The process is CPU bound.

I am pretty sure the improvement seen in the prfiler is repeatable and
less than the noise in the profiling, so why don't the timeit times
change?  The only reason I can think of is that the profiler is blocking
some kind of optimisation (which seems odd, since that makes profiling
somewhat pointless).

Full disclosure - the code linked above is not quite identical to my
current workspace (I have replaced the append/pop with an explicit index
into an array whose size is managed separately).

Andrew

--
http://mail.python.org/mailman/listinfo/python-list

Re: Roulette wheel

[Peter Otten]

mattia wrote:

>> Note how get_roulette_wheel() is now completeley independent of the
>> concrete problem you are using it for.
> 
> Ok, but also a lot more memory consuming ;-)

I don't think so. Python references objects; therefore the list

[tiny_little_thing]*N 

does not consume more memory than 

[big_fat_beast]*N

Or did you have something else in mind?

Peter

--
http://mail.python.org/mailman/listinfo/python-list

wx, qt, gtk

[Stefano]

Hello everybody

After many little programs now I must prepare a serious desktop application 
.
Without discussing about licensing but only from the technical point of view 
I have tried wx,wt,gtk graphical gui to build 'normal gui apps ' and 
everyone goes well.
In the end of all  i searched in internet and i've found that applications 
( even commercial ) written with gtk are more and more than other written 
with wx and qt (not only with python)


Any commets are appreciated

Stefano

--
http://mail.python.org/mailman/listinfo/python-list

Re: Can't override class |__new__

[jelle feringa]

Hi Aaron,

Thanks so much for your feedback.

> Regardless of CGAL's dictionary, it instantiates a Facet.

True, when I add attributes to it, they are disregarded when looping
through the facets later on.

> Depending on the details, you may need only to cut-and-paste your own
> 'make_triangle' function, and just replace the line I showed.

Ok, thanks so much for pointing me in the right direction.
I suppose there is no way to fix this on the interpreter level...
Thanks so much for your help in finding this bug!

> Do you have a link to the source for CGAL?

In fact I'm using the python wrappers of CGAL.
Here's a link to the Polyhedron module.
In fact, I can add methods and attributes to this class, just the Facet 
instances
do not behave as expected.

https://gforge.inria.fr/plugins/scmsvn/viewcvs.php/trunk/cgal-
python/bindings/Polyhedron/Py_Polyhedron_3.cpp?rev=167&root
=cgal-python&view=markup

Again, thanks for you help,

-jelle

--
http://mail.python.org/mailman/listinfo/python-list

Number of Packages in the "cheeseshop"

[Michael Rudolf]

Hi, I just wondered how many Packages are in the Python Package Index.

I could not find any counter, but I found that there is a category
overview on http://pypi.python.org/pypi?%3Aaction=browse .

A quick look at the HTML told me that the number of Packages per
Category is listed surrounded by parentheses, at most one per line.

So I counted them:

import urllib
sum=0
for t in urllib.urlopen('http://pypi.python.org/pypi?%3Aaction=browse'):
t=t.split('(')[-1].split(')')[0]
try:
sum += int(t)
except ValueError:
pass # print "OMG cannot convert %s to int" % t
print "sum is: %s" % sum 

Which yields: sum is: 31670

That would be around half the weight of CPAN, which would be a
not-so-bad result ;)

My Questions: 
a) Are there package listed in multiple Categories, which would breaking
my counting?
b) Did I make some other mistake(s)?
c) is there a counter which yields the current number of PyPI-Packages?

PS: Please excuse my bad english, I am not a native speaker.

THX, Michael
--
http://mail.python.org/mailman/listinfo/python-list

Re: Response codes and \r\n

[Peter Otten]

Catherine Heathcote wrote:

> I am reading an XML file (code at the end if it helps) and all goes well
> except I am getting the http response code printed. So everything (hat
> works of course) has "200 OK" on the first line. Am I missing some
> simple  way of surprising this, or should I just delete the 1st line
> before playing with the content?

Just remove the line

print(response.status, response.reason)

Whoever wrote that script *wanted* it to print that "200 OK"

> Also everything I get has "\r\n" in it, which atm I am getting rid of
> with strip(), is that the best way?

That is because you are getting back bytes, not a string. You can convert
the data with 

data = data.decode("utf-8") # or whatever the actual encoding is

and then mess around with string methods or regular expressions, but if you
want to do it right you have to learn about element tree

http://docs.python.org/3.0/library/xml.etree.elementtree.html#module-xml.etree.ElementTree

or one of the alternatives.

Also note that conn.close will *not* call the close method. It should be

conn.close()

Peter
--
http://mail.python.org/mailman/listinfo/python-list

Re: Response codes and \r\n

[Marc 'BlackJack' Rintsch]

On Thu, 05 Mar 2009 09:37:49 +, Catherine Heathcote wrote:

> I am reading an XML file (code at the end if it helps) and all goes well
> except I am getting the http response code printed. So everything (hat
> works of course) has "200 OK" on the first line. Am I missing some
> simple  way of surprising this, or should I just delete the 1st line
> before playing with the content?

Is this line actually part of `data`!?  I would guess it is printed in 
line 22, so it is not part of `data` and there is no need to delete it.

> Also everything I get has "\r\n" in it, which atm I am getting rid of
> with strip(), is that the best way?

At which point do you get rid of it and why?

BTW the last line of the code snippet needs parenthesis to actually 
*call* the `conn.close` method.

Ciao,
Marc 'BlackJack' Rintsch
--
http://mail.python.org/mailman/listinfo/python-list

Re: Response codes and \r\n

[Richard Brodie]

"Catherine Heathcote"  wrote in message 
news:n3nrl.2951$lc7.2...@text.news.virginmedia.com...
=
> I am reading an XML file (code at the end if it helps) and all goes well 
> except I am 
> getting the http response code printed.

I suggest you comment out line 22. The status shouldn't be in the data.

> Also everything I get has "\r\n" in it, which atm I am getting rid of with 
> strip(), is 
> that the best way?

I would use and XML parser such as Elementtree, and let it handle it.
Resist the temptation to think "it's a simple format, I'll parse it myself".
Otherwise strip() or rstrip('\r\n') is fine, depending how much whitespace
matters.

> conn.close

Note that statement does nothing, it's not the same as conn.close() 


--
http://mail.python.org/mailman/listinfo/python-list

Re: FW: Anyone read "Python Interview Questions: Python Certification Review"?

[Paul Sammy]

Thanks for the comments on this topic.  I shall read the thread suggest by
Tim and avoid the book.  (I did look for reviews on Amazon, but I used
amazon.co.uk, so I missed the Perl book reviews).
I also emailed the company to ask more about the 'Python Certification' that
the book title alluded to; if I get a reply I will forward it to the mailing
list.

thanks again,

Paul

2009/3/4 Grimes, George 

> The listing for the book on Amazon did not have any reviews but they had an
> almost identical page for their Perl book.  It had 3 reviews giving one star
> (the lowest rating) each.  All reported that it was just the reproduction of
> on-list postings, no original material, and no attempt to make a coherent
> presentation.
>
> It sounds like the sort of thing to avoid.  I'm just learning Python myself
> and I'll look elsewhere.
>
> George
>
>
> George A. Grimes
> 972-995-0190 - Desk
> 214-205-0244 - Cell
>
> "The major difference between a thing that might go wrong and a thing that
> cannot possibly go wrong is that when a thing that cannot possibly go wrong
> goes wrong it usually turns out to be impossible to get at or repair." --
> Douglas Adams
> -Original Message-
> From: Steve Holden [mailto:st...@holdenweb.com]
> Sent: Wednesday, March 04, 2009 8:44 AM
> To: python-list@python.org
> Subject: Re: Anyone read "Python Interview Questions: Python Certification
> Review"?
>
> Paul Sammy wrote:
> > Hi,
> >
> > On a recent trawl of the internet for some Python books, I came across
> > "Python Interview Questions: Python Certification Review" by
> > ITCOOKBOOK.COM 
> >
> >
> http://www.itcookbook.com/store/index.php?main_page=product_info&products_id=15
> > <
> http://www.itcookbook.com/store/index.php?main_page=product_info&products_id=15
> >
> >
> > Has anyone used this, or even one of the related books?  I can find NO
> > reviews of any of these books online, and the page on the Java version
> > of the book has factual errors (the blurb talks about operator
> > overloading and pointers, neither of which Java has).
> >
> > Furthermore the ITCOOKBOOK homepage appears to selling some sort of
> > "20-step plan" which leads to suspiciously large amounts of money -
> > after paying them money of course (
> http://itcookbook.com/online_training/).
> >
> > Anyway, thought I would run this past you guys in case I'm being unduly
> > suspicious or cautious.
> >
> As far as I know there isn't anything like a useful Python certification
> in the marketplace at the moment, and this book certainly doesn't seem
> to be linked with one.
>
> The fact that they don't say how or whether you get certified would
> increase my suspicion level, and the 20-step plan appears to be a simple
> come-on with not much behind it.
>
> I am hoping that when O'Reilly School of Technology publish their Python
> classes (which I am currently busy writing) this situation will change.
> There will be a university-associated certification available for those.
>
> regards
>  Steve
> --
> Steve Holden+1 571 484 6266   +1 800 494 3119
> Holden Web LLC  http://www.holdenweb.com/
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
http://mail.python.org/mailman/listinfo/python-list

Re: Roulette wheel

[mattia]

Il Thu, 05 Mar 2009 10:46:58 +0100, Peter Otten ha scritto:

> mattia wrote:
> 
>>> Note how get_roulette_wheel() is now completeley independent of the
>>> concrete problem you are using it for.
>> 
>> Ok, but also a lot more memory consuming ;-)
> 
> I don't think so. Python references objects; therefore the list
> 
> [tiny_little_thing]*N
> 
> does not consume more memory than
> 
> [big_fat_beast]*N
> 
> Or did you have something else in mind?
> 
> Peter

Ok, understood. So if I have e.g. [[200 elements]]*N, then I'll have N 
pointers to the same location of my seq, right?
--
http://mail.python.org/mailman/listinfo/python-list

problem with pydoc under python 2.6.1

[steve . ferg . bitbucket]

Has anybody encountered problems running pydoc with version 2.6.1?
I'm getting an error message that pydoc cannot import namedtuple
(details below).
(I'm running under 64-bit Windows Vista, although that probably is not
important.)

Here's my batch file, pydoc_test.bat:
 =
 @echo on
 set pyver=python25
 python c:\%pyver%\Lib\pydoc.py -w easygui

 set pyver=python26
 python c:\%pyver%\Lib\pydoc.py -w easygui
 =

Here's what I get:
 =
 c:\pydev\easygui\v086>pydoc_test.bat

 c:\pydev\easygui\v086>set pyver=python25

 c:\pydev\easygui\v086>python c:\python25\Lib\pydoc.py -w easygui
 wrote easygui.html

 c:\pydev\easygui\v086>set pyver=python26

 c:\pydev\easygui\v086>python c:\python26\Lib\pydoc.py -w easygui
 Traceback (most recent call last):
   File "c:\python26\Lib\pydoc.py", line 55, in 
 import sys, imp, os, re, types, inspect, __builtin__, pkgutil
   File "c:\python26\Lib\inspect.py", line 42, in 
 from collections import namedtuple
 ImportError: cannot import name namedtuple
 

-- Steve Ferg
--
http://mail.python.org/mailman/listinfo/python-list

Re: problem with pydoc under python 2.6.1

[yinoneh]

On Mar 5, 1:29 pm, steve.ferg.bitbuc...@gmail.com wrote:
> Has anybody encountered problems running pydoc with version 2.6.1?
> I'm getting an error message that pydoc cannot import namedtuple
> (details below).
> (I'm running under 64-bit Windows Vista, although that probably is not
> important.)
>
> Here's my batch file, pydoc_test.bat:
>  =
> �...@echo on
>  set pyver=python25
>  python c:\%pyver%\Lib\pydoc.py -w easygui
>
>  set pyver=python26
>  python c:\%pyver%\Lib\pydoc.py -w easygui
>  =
>
> Here's what I get:
>  =
>  c:\pydev\easygui\v086>pydoc_test.bat
>
>  c:\pydev\easygui\v086>set pyver=python25
>
>  c:\pydev\easygui\v086>python c:\python25\Lib\pydoc.py -w easygui
>  wrote easygui.html
>
>  c:\pydev\easygui\v086>set pyver=python26
>
>  c:\pydev\easygui\v086>python c:\python26\Lib\pydoc.py -w easygui
>  Traceback (most recent call last):
>    File "c:\python26\Lib\pydoc.py", line 55, in 
>      import sys, imp, os, re, types, inspect, __builtin__, pkgutil
>    File "c:\python26\Lib\inspect.py", line 42, in 
>      from collections import namedtuple
>  ImportError: cannot import name namedtuple
>  
>
> -- Steve Ferg

the same for Python trunk (develompment version), on Ubuntu 8.04
I'm filing a bug report...
--
http://mail.python.org/mailman/listinfo/python-list

Re: Roulette wheel

[Peter Otten]

mattia wrote:

> Il Thu, 05 Mar 2009 10:46:58 +0100, Peter Otten ha scritto:
> 
>> mattia wrote:
>> 
 Note how get_roulette_wheel() is now completeley independent of the
 concrete problem you are using it for.
>>> 
>>> Ok, but also a lot more memory consuming ;-)
>> 
>> I don't think so. Python references objects; therefore the list
>> 
>> [tiny_little_thing]*N
>> 
>> does not consume more memory than

Oops, should have been less.

>> 
>> [big_fat_beast]*N
>> 
>> Or did you have something else in mind?
>> 
>> Peter
> 
> Ok, understood. So if I have e.g. [[200 elements]]*N, then I'll have N
> pointers to the same location of my seq, right?

Right. You can verify this with

>>> v = [0]
>>> items = [v]*5
>>> v[0] = 42
>>> items
[[42], [42], [42], [42], [42]]

which often surprises newbies.

Peter
--
http://mail.python.org/mailman/listinfo/python-list

Re: Can't override class |__new__

[Lie Ryan]

jelle feringa wrote:

Hi,

I'm working with a C++ module ( CGAL, comp.geom. with exact arithmic )
and am having troubles finding a way to override how the modules returns
objects. What I'm trying to do is to extend the Facet class, but when I try
to use my version of the class, the parent class is still being returned

import CGAL

CGAL.Facet = OtherFacet
CGAL.Polyhedron.Facet = OtherFacet

p = CGAL.Polyhedron_3()


You're not creating Facet object here, not even Polyhedron.Facet.


p.make_triangle()

for f in p.facets:
print f


# Here I was expecting a OtherFacet object

Is there a way of getting around this, or is this behaviour hardcoded in the
C++ module?


I think so. External module written in C/C++ is often statically linked.


Can I perhaps alter it using  the new module or __new__ method?

Many thanks in advance,

-jelle


--
http://mail.python.org/mailman/listinfo/python-list

Re: problem with pydoc under python 2.6.1

[Yinon Ehrlich]

On Mar 5, 1:48 pm, yino...@gmail.com wrote:
> On Mar 5, 1:29 pm, steve.ferg.bitbuc...@gmail.com wrote:
>
>
>
> > Has anybody encountered problems running pydoc with version 2.6.1?
> > I'm getting an error message that pydoc cannot import namedtuple
> > (details below).
> > (I'm running under 64-bit Windows Vista, although that probably is not
> > important.)
>
> > Here's my batch file, pydoc_test.bat:
> >  =
> > �...@echo on
> >  set pyver=python25
> >  python c:\%pyver%\Lib\pydoc.py -w easygui
>
> >  set pyver=python26
> >  python c:\%pyver%\Lib\pydoc.py -w easygui
> >  =
>
> > Here's what I get:
> >  =
> >  c:\pydev\easygui\v086>pydoc_test.bat
>
> >  c:\pydev\easygui\v086>set pyver=python25
>
> >  c:\pydev\easygui\v086>python c:\python25\Lib\pydoc.py -w easygui
> >  wrote easygui.html
>
> >  c:\pydev\easygui\v086>set pyver=python26
>
> >  c:\pydev\easygui\v086>python c:\python26\Lib\pydoc.py -w easygui
> >  Traceback (most recent call last):
> >    File "c:\python26\Lib\pydoc.py", line 55, in 
> >      import sys, imp, os, re, types, inspect, __builtin__, pkgutil
> >    File "c:\python26\Lib\inspect.py", line 42, in 
> >      from collections import namedtuple
> >  ImportError: cannot import name namedtuple
> >  
>
> > -- Steve Ferg
>
> the same for Python trunk (develompment version), on Ubuntu 8.04
> I'm filing a bug report...

sorry, it's OK.
My fault was using installed python to run development version's
library.
not sure about python26 for Windows...
Yinon
--
http://mail.python.org/mailman/listinfo/python-list

Re: Dr Dobbs' Python Weekly URL Archive?

[Steve Holden]

Hendrik van Rooyen wrote:
>  "Steve Holden"  wrote:
>> My well-known-search-engine-foo must be at an all-time low today. *Is*
>> there an index and I can't see for looking?
>>
> typing in python weekly at google gives me:
> 
> Python-URL!The bookmark for this page is:
> http://purl.org/thecliff/python/url.html. Dr. Dobb's Python-URL! - 2000/07/18 
> by
> Andrew M. Kuchling; Dr. Dobb's Python-URL! ...
> www.equi4.com/wikis/urls/82 - 9k - Cached - Similar pages
> 
> This is a link to:
> 
> http://www.equi4.com/wikis/urls/82
> 
Well, I was hoping for something more recent than July 20, 2000. When I
said my google-fu was low I didn't mean so low that I was incapable of
entering "dr dobbs python url archive" ... but thanks anyway!

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/

--
http://mail.python.org/mailman/listinfo/python-list

how to convert from network to host byte order

[Evan]

Hello ~

I'm new with python,  what my problem is, I have a binary file, I want
to read first 2 bytes and convert it to host byte order, then write it
to another file.

I try to use 'socket' and 'struct', but somehow i can not get it
working fine:

for example, totally I'm not sure if my steps is correct or not:

>>> import socket
>>> f=open('a.bin','rb')
>>> f.read(2)
'\x04\x00'

>>> f.seek(0)
>>> st=f.read(2)
>>> e=open('test.bin','w+b')
>>> e.write(socket.ntohs(struct.unpack('H',st[:2])[0]))
Traceback (most recent call last):
  File "", line 1, in 
TypeError: argument 1 must be string or read-only buffer, not int
+

It failed due to the parameter is 'int', not 'str' in write function.
but how can i do that?

Thanks,
Evan
--
http://mail.python.org/mailman/listinfo/python-list

Re: String Identity Test

[Bruno Desthuilliers]

Hendrik van Rooyen a écrit :

"S Arrowsmith"  wrote:


"Small" integers get a similar treatment:


a = 256
b = 256
a is b

True

a = 257
b = 257
a is b

False


This is weird - I would have thought that the limit
of "small" would be at 255 - the biggest number to 
fit in a byte.  256 takes two bytes, so it must be

an arbitrary limit


It is, and has changed from version to version.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Can't override class |__new__

[jelle feringa]

> > CGAL.Facet = OtherFacet
> > CGAL.Polyhedron.Facet = OtherFacet
> > p = CGAL.Polyhedron_3()

> You're not creating Facet object here, not even Polyhedron.Facet.

Right, which is not the point; I'm trying to override the Facet, a topological
 entity of which a Polyhedron is composed of .
( vertex -> halfedge -> facet -> polyhedron )
 
> I think so. External module written in C/C++ is often statically linked.

I'm sorry, but I don't follow you; what has linking to do with changing
 the behaviour of the compiled module?

Thanks,

-jelle

--
http://mail.python.org/mailman/listinfo/python-list

Re: String Identity Test

[Bruno Desthuilliers]

Avetis KAZARIAN a écrit :


> Well, it's not about curiosity, it's more about performance.


Steve Holden wrote:

(snip)

So, don't try to translate concepts from one language to another.


I'll try ;]


Also and FWIW:

1/ Python has some very handy tools when it comes to perfs - like a 
couple profilers (to identify bottlenecks), or the timeit module (for 
quick benchmarks).


2/ Most "best practice" idioms are frequently discussed here

3/ If you have performance problems related to wrong algorithm/data 
structure, some of us here _really_ enjoy helping !-)


Welcome onboard.
--
http://mail.python.org/mailman/listinfo/python-list

Re: how to convert from network to host byte order

[Peter Otten]

Evan wrote:

> Hello ~
> 
> I'm new with python,  what my problem is, I have a binary file, I want
> to read first 2 bytes and convert it to host byte order, then write it
> to another file.
> 
> I try to use 'socket' and 'struct', but somehow i can not get it
> working fine:
> 
> for example, totally I'm not sure if my steps is correct or not:
> 
 import socket
 f=open('a.bin','rb')
 f.read(2)
> '\x04\x00'
> 
 f.seek(0)
 st=f.read(2)
 e=open('test.bin','w+b')
 e.write(socket.ntohs(struct.unpack('H',st[:2])[0]))
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: argument 1 must be string or read-only buffer, not int
> +
> 
> It failed due to the parameter is 'int', not 'str' in write function.
> but how can i do that?
> 
> Thanks,
> Evan

I think for just 2 bytes I'd do it manually:

a = open(...).read(2)
if sys.byteorder == "little":
a = a[::-1]
open(...).write(a)

For larger chunks of data I'd use a byte array and the byteswap() method,
see

http://docs.python.org/library/array.html

Peter
--
http://mail.python.org/mailman/listinfo/python-list

Response codes and \r\n

[Catherine Heathcote]

Heya,
I am reading an XML file (code at the end if it helps) and all goes well 
except I am getting the http response code printed. So everything (hat 
works of course) has "200 OK" on the first line. Am I missing some 
simple  way of surprising this, or should I just delete the 1st line 
before playing with the content?


Also everything I get has "\r\n" in it, which atm I am getting rid of 
with strip(), is that the best way?


http://pastebin.com/m4120242

Thanks
--
http://mail.python.org/mailman/listinfo/python-list

Re: siple for in expression

[Matko]

Thanks everyone!

Matko ;)

Thanks
"Matko"  wrote in message 
news:gojkfo$2p...@gregory.bnet.hr...
> Hello!
>
> Can someone help me to understand the following code:
>
> uv_face_mapping = [[0,0,0,0] for f in faces]
>
> Thank You very much!
>
> Matko from Croatia
> 


--
http://mail.python.org/mailman/listinfo/python-list

RE: Help required to read and print lines based on the type of first character

[rdmurray]

abhinayaraj.r...@emulex.com wrote:
> Thank you for the suggestions.
> 
> Some little reading gave the idea and it works well too. :)
> 
> Here is the code:
> fileIN = open("test.txt")
> count = 0
> for line in fileIN:
>   data= line
> 
>   if '' in data:
>   count = 4
>   elif '###' in data:
>   count = 3
>   elif '##' in data:
>   count = 2
>   elif '#' in data:
>   count = 1
>   elif data.find('#') == -1:
>   count = 0
> 
>   if (count == 0):
>   print data + '\nlooks like a code line...\n'
>   elif(count== 4):
>   print data + '\ninvalid line!\n'
>   elif count>=1:
>   for i in range(0, count):
>   print data

What happens when your input file has a line like:

This is a ## fake line

You might want to check out the strip and startswith methods of strings.

--RDM

--
http://mail.python.org/mailman/listinfo/python-list

Re: Help required to read and print lines based on the type of first character

[Bruno Desthuilliers]

(answering to the OP)

En Wed, 04 Mar 2009 07:36:01 -0200,  escribió:


I am a beginner in Python. In fact, beginner to coding/ scripting.

Here is a scenario, I need to code. Need your help on this:


Your first task here should be to refine the specs - too much 
ambiguities in it:



A script that
1.  Reads from a file (may be a local file say test.txt)


reads what ? The whole content ? A (yet unspecified) portion ? etc...


2.  And, if the line


which line ?

begins with a "#", should print the line one 
time


" yadda" begins with a "#". So according to this rule, it should be 
printed once.



3.  if the line has "##", should print the line 2 times,


" yadda" 'has' (contains) "##", so it should be printed twice. But 
it also _begins_ with a '#', so according to rule 2, it should be 
printed once. Is the 'one time' in rule 2 supposed to mean 'at least 
onces', or 'once and only once' ? In this last case, rule 3 contradicts 
rule 2.


Also, "yadday ## woops" 'has' (contains) "##". According to rule 3, it 
should be printed twice. Is that right ?


4.  And, if there are "###" in the beginning of the line, should 
print the same line 3times,


" yadda" begins with "###", so it should be printed thrice. It also 
contains "##" (cf rule 3) and starts with "#" (cf rule 2). How is this 
rule supposed to be understood ?


5.  And, if the line contains "" or more "#"'s, then print as 
"an invalid line"


" yadda" starts with '#', contains '##', and starts with '###'. 
Which rule is supposed to apply here ?



6.  if the line contains no "#" then print "looks like a code line"



The first step in programming is to get accurate, unambigous and 
well-expressed specs. In the above case (I mean, at this level of 
detail), "accurate, unambigous and well-expressed specs" are almost 
pseudocode. Doing a bit of mind-reading (which is certainly *not* the 
right thing to do - in real life, I'd just go back to the customer or 
whoever handed me such specs to sort this out), I came out with the 
following rewrite:


1. open a given file in text mode
2. read it line by line
3. for each line:
3.1. if the line contains/startswith (?) more than three '#':
print "an invalid line"
3.2. if the line starts with one, two or three '#':
 print as many times the line as there are '#'
3.2 else (imply : the line contains no '#'):
   print "looks like a code line"

So basically, you have your algorithm. Now you just need to find out how 
to do each of these tasks in Python. That is :


a. how to open a file for reading in text mode (NB: 'text mode' may or 
not makes sense, according to the OS)


b. how to  iterate over the lines in this file once it's correctly opened

c. how to test for the presence and position of a given character / 
substring in a string


d. how to "print" something from your program.


Good news: all this is pretty well documented. I'd say that the most 
"tricky" part is c., since there's more than one possible solution, but 
since it's about strings, looking for what features Python strings has 
to offer, and trying them out in the interactive interpreter should 
solve the problem very quickly.



HTH
--
http://mail.python.org/mailman/listinfo/python-list

Re: Can't override class |__new__

[Lie Ryan]

jelle feringa wrote:

CGAL.Facet = OtherFacet
CGAL.Polyhedron.Facet = OtherFacet
p = CGAL.Polyhedron_3()



You're not creating Facet object here, not even Polyhedron.Facet.


Right, which is not the point; I'm trying to override the Facet, a topological
 entity of which a Polyhedron is composed of .
( vertex -> halfedge -> facet -> polyhedron )
 

I think so. External module written in C/C++ is often statically linked.


I'm sorry, but I don't follow you; what has linking to do with changing
 the behaviour of the compiled module?


Aaron explained it better than I did. Basically the C/C++ module uses 
C/C++ version of Facet class directly without consulting python 
interpreter, this is what I meant by static linking.


Since python is dynamic language, all reference is dynamic, but C/C++ 
are static language, extensions written in C/C++ have to make extra 
effort to create dynamic reference.

--
http://mail.python.org/mailman/listinfo/python-list

Re: how to convert from network to host byte order

[Joe Riopel]

On Thu, Mar 5, 2009 at 7:26 AM, Evan  wrote:
> Hello ~
>
> I'm new with python,  what my problem is, I have a binary file, I want
> to read first 2 bytes and convert it to host byte order, then write it
> to another file.

Have you checked out socket.htons, socket.ntohs, etc ?
--
http://mail.python.org/mailman/listinfo/python-list

Re: wx, qt, gtk

[sturlamolden]

On Mar 5, 11:11 am, "Stefano"  wrote:

> In the end of all  i searched in internet and i've found that applications
> ( even commercial ) written with gtk are more and more than other written
> with wx and qt (not only with python)


>From a technical point of view, Qt is the superior toolkit. Licensing
has until now been an impediment. Licensing is still an impediment for
using PyQt, but not Qt.



--
http://mail.python.org/mailman/listinfo/python-list

Re: Number of Packages in the "cheeseshop"

[John Machin]

On Mar 5, 9:40 pm, Michael Rudolf  wrote:
> Hi, I just wondered how many Packages are in the Python Package Index.
>
> I could not find any counter,

Main page (http://pypi.python.org/pypi), right at the top:
"""
The Python Package Index is a repository of software for the Python
programming language. There are currently 5883 packages here.
"""

The devs must have read your posting and slammed in a quick fix ;-)

> but I found that there is a category
> overview onhttp://pypi.python.org/pypi?%3Aaction=browse.
>
> A quick look at the HTML told me that the number of Packages per
> Category is listed surrounded by parentheses, at most one per line.
>
> So I counted them:
>
> import urllib
> sum=0
> for t in urllib.urlopen('http://pypi.python.org/pypi?%3Aaction=browse'):
>     t=t.split('(')[-1].split(')')[0]

That statement is a thing of beauty and a joy forever. I wonder what
it does.

>     try:
>         sum += int(t)
>     except ValueError:
>         pass # print "OMG cannot convert %s to int" % t
> print "sum is: %s" % sum
>
> Which yields: sum is: 31670
>
> That would be around half the weight of CPAN, which would be a
> not-so-bad result ;)
>
> My Questions:
> a) Are there package listed in multiple Categories, which would breaking
> my counting?

Next you'll be asking if items are listed in multiple categories on
eBay :-)

Have you considered looking at the listing for some individual
packages? Here's one:

#  Categories

* Development Status :: 5 - Production/Stable
* Intended Audience :: Developers
* License :: OSI Approved :: BSD License
* Operating System :: OS Independent
* Programming Language :: Python
* Topic :: Database
* Topic :: Internet :: WWW/HTTP :: Dynamic Content :: CGI Tools/
Libraries
* Topic :: Office/Business :: Financial :: Spreadsheet
* Topic :: Software Development :: Libraries :: Python Modules

So that's 9 categories. And 4 topics -- that'd be "keyword spamming"
on eBay :-)

> b) Did I make some other mistake(s)?

Yes.

> c) is there a counter which yields the current number of PyPI-Packages?

Yes.
>
> PS: Please excuse my bad english, I am not a native speaker.    
>
> THX, Michael

--
http://mail.python.org/mailman/listinfo/python-list

Re: how to convert from network to host byte order

[Philipp Hagemeister]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Evan wrote:
>>> inp='\x04\x00'
>>> out = socket.ntohs(struct.unpack('H',inp[:2])[0]))
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: argument 1 must be string or read-only buffer, not int

Your approach is nearly right. First of all, you have to tell
struct.unpack it should unpack from network order ("!"):

>>> struct.unpack('!H', inp)[0]
1024

Then you want to repack it in host byte order. Use "=" for that.

>>> out = struct.pack('=H', struct.unpack('!H', inp)[0])
>>> out
'\x00\x04'

For more information, look for "Size and alignment" in
http://docs.python.org/library/struct.html.

Regards,

Philipp Hagemeister
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEAREKAAYFAkmv2JkACgkQ9eq1gvr7CFymKACghFXMZb9D6pkWZQdapvwTsKJ5
b0UAn0Uvbcguv/rdxjFKXhMQz22+Notn
=ZiKx
-END PGP SIGNATURE-
--
http://mail.python.org/mailman/listinfo/python-list

Re: PyPI editing

[David Stanek]

On Thu, Mar 5, 2009 at 3:32 AM, andrew cooke  wrote:
>
> Apparently not (this will probably change, but see
> http://pypi.python.org/pypi/LEPL/2.0.1 which is currently displaying
> restructured text literally)
>

Did you put the ReST in the description or the long_description? Take
a look at the setup.py from the example you gave:
http://code.google.com/p/pypp/source/browse/trunk/setup.py.

Warning: I have not done this myself.

-- 
David
http://www.traceback.org
--
http://mail.python.org/mailman/listinfo/python-list

Re: 2to3 Help?

[Ant]

On Jan 15, 6:43 am, "James Mills" 
wrote:
...
> That is a good idea :) Windows teaches you nothing!

Not true. Windows teaches you the art of the workaround...
--
http://mail.python.org/mailman/listinfo/python-list

Re: PyPI editing

[andrew cooke]

ah!  thanks very much.  that is obviously what i need to do.

i am pretty sure i was doing the wrong thing, but am now at work.  i'll
check this tonight.

thanks again,
andrew


David Stanek wrote:
> On Thu, Mar 5, 2009 at 3:32 AM, andrew cooke  wrote:
>>
>> Apparently not (this will probably change, but see
>> http://pypi.python.org/pypi/LEPL/2.0.1 which is currently displaying
>> restructured text literally)
>>
>
> Did you put the ReST in the description or the long_description? Take
> a look at the setup.py from the example you gave:
> http://code.google.com/p/pypp/source/browse/trunk/setup.py.
>
> Warning: I have not done this myself.
>
> --
> David
> http://www.traceback.org
>
>


--
http://mail.python.org/mailman/listinfo/python-list

Re: how to convert from network to host byte order

[Evan]

On Mar 5, 9:50 pm, Philipp Hagemeister  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA512
>
> Evan wrote:
> >>> inp='\x04\x00'
> >>> out = socket.ntohs(struct.unpack('H',inp[:2])[0]))
> > Traceback (most recent call last):
> >   File "", line 1, in 
> > TypeError: argument 1 must be string or read-only buffer, not int
>
> Your approach is nearly right. First of all, you have to tell
> struct.unpack it should unpack from network order ("!"):
>
> >>> struct.unpack('!H', inp)[0]
>
> 1024
>
> Then you want to repack it in host byte order. Use "=" for that.
>
> >>> out = struct.pack('=H', struct.unpack('!H', inp)[0])
> >>> out
>
> '\x00\x04'
>
> For more information, look for "Size and alignment" 
> inhttp://docs.python.org/library/struct.html.
>
> Regards,
>
> Philipp Hagemeister
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.9 (GNU/Linux)
>
> iEYEAREKAAYFAkmv2JkACgkQ9eq1gvr7CFymKACghFXMZb9D6pkWZQdapvwTsKJ5
> b0UAn0Uvbcguv/rdxjFKXhMQz22+Notn
> =ZiKx
> -END PGP SIGNATURE-

That's good, thanks Philipp
--
http://mail.python.org/mailman/listinfo/python-list

Re: Number of Packages in the "cheeseshop"

[Michael Rudolf]

Am Thu, 5 Mar 2009 05:38:58 -0800 (PST)
schrieb John Machin :

> Main page (http://pypi.python.org/pypi), right at the top:
> """
> The Python Package Index is a repository of software for the Python
> programming language. There are currently 5883 packages here.
> """

Ooops... totally missed that... must have been blind, sorry.
Thank you.

> > for t in \
> > urllib.urlopen('http://pypi.python.org/pypi?%3Aaction=browse'):
> > t=t.split('(')[-1].split(')')[0]
> That statement is a thing of beauty and a joy forever. I wonder what
> it does.
extracts everything between parentheses per line, as long as there is
exactly one '(' and one ')' in it (true for that site).

Didnt want to parse the HTML or write a regex for that simple Job.

Anyways, sorry for that stupid post and thanks for pointing out that
there actually *is* a counter.
Next time I will readjust my caffeine-in-blood-level before posting. ;)

Michael
--
http://mail.python.org/mailman/listinfo/python-list

Re: Dr Dobbs' Python Weekly URL Archive?

[Mike Driscoll]

On Mar 5, 6:15 am, Steve Holden  wrote:
> Hendrik van Rooyen wrote:
> >  "Steve Holden"  wrote:
> >> My well-known-search-engine-foo must be at an all-time low today. *Is*
> >> there an index and I can't see for looking?
>
> > typing in python weekly at google gives me:
>
> > Python-URL!The bookmark for this page is:
> >http://purl.org/thecliff/python/url.html. Dr. Dobb's Python-URL! - 
> >2000/07/18 by
> > Andrew M. Kuchling; Dr. Dobb's Python-URL! ...
> >www.equi4.com/wikis/urls/82- 9k - Cached - Similar pages
>
> > This is a link to:
>
> >http://www.equi4.com/wikis/urls/82
>
> Well, I was hoping for something more recent than July 20, 2000. When I
> said my google-fu was low I didn't mean so low that I was incapable of
> entering "dr dobbs python url archive" ... but thanks anyway!
>
> regards
>  Steve
> --
> Steve Holden           +1 571 484 6266   +1 800 494 3119
> Holden Web LLC                http://www.holdenweb.com/
> Want to know? Come to PyCon - soon!http://us.pycon.org/

Perhaps you're looking for this: 
http://www.ddj.com/TechSearch/searchResults.jhtml?queryText=python

Gabriel has been including the link in his Python roundup weekly
email...

Mike
--
http://mail.python.org/mailman/listinfo/python-list

Re: Reading a file

[MRAB]

Aahz wrote:

In article ,
Terry Reedy   wrote:

for line in open('char.txt'):
  if line.find('sweet') != -1 or line.find('blue') != -1:
print(line)


For any recent Python, this should be:

if 'sweet' in line or 'blue' in line:

Although I think that for the OP's use case, it ought to be:

if line.startswith('sweet=') or line.startswith('blue=')

Or:

if line.startswith(('sweet=', 'blue=')):

--
http://mail.python.org/mailman/listinfo/python-list

RE: Help required to read and print lines based on the type of first character

[Abhinayaraj . Raju]

Thank you so much for your guidance, Bruno.

This should help me in a long way.

Here is the code I have written.

path = raw_input("\nEnter location eg. c:/buffer/test.txt : \n") 
fileIN = open(path)
count = 0
for line in fileIN:
data= line

if '' in data:
count = 4
elif '###' in data:
count = 3
elif '##' in data:
count = 2
elif '#' in data:
count = 1
elif data.find('#') == -1:
count = 0


if (count == 0 and data!=""):
print data + '\nlooks like a code line...\n'
elif(count== 4):
print data + '\ninvalid line!\n'
elif count>=1:
for i in range(0, count):
print data
Thanks
Abhi




-Original Message-
From: Bruno Desthuilliers [mailto:bruno.42.desthuilli...@websiteburo.invalid] 
Sent: Thursday, March 05, 2009 6:50 PM
To: python-list@python.org
Subject: Re: Help required to read and print lines based on the type of first 
character

(answering to the OP)
> En Wed, 04 Mar 2009 07:36:01 -0200,  escribió:
> 
>> I am a beginner in Python. In fact, beginner to coding/ scripting.
>>
>> Here is a scenario, I need to code. Need your help on this:

Your first task here should be to refine the specs - too much 
ambiguities in it:

>> A script that
>> 1.  Reads from a file (may be a local file say test.txt)

reads what ? The whole content ? A (yet unspecified) portion ? etc...

>> 2.  And, if the line

which line ?

>> begins with a "#", should print the line one 
>> time

" yadda" begins with a "#". So according to this rule, it should be 
printed once.

>> 3.  if the line has "##", should print the line 2 times,

" yadda" 'has' (contains) "##", so it should be printed twice. But 
it also _begins_ with a '#', so according to rule 2, it should be 
printed once. Is the 'one time' in rule 2 supposed to mean 'at least 
onces', or 'once and only once' ? In this last case, rule 3 contradicts 
rule 2.

Also, "yadday ## woops" 'has' (contains) "##". According to rule 3, it 
should be printed twice. Is that right ?

>> 4.  And, if there are "###" in the beginning of the line, should 
>> print the same line 3times,

" yadda" begins with "###", so it should be printed thrice. It also 
contains "##" (cf rule 3) and starts with "#" (cf rule 2). How is this 
rule supposed to be understood ?

>> 5.  And, if the line contains "" or more "#"'s, then print as 
>> "an invalid line"

" yadda" starts with '#', contains '##', and starts with '###'. 
Which rule is supposed to apply here ?

>> 6.  if the line contains no "#" then print "looks like a code line"
>>

The first step in programming is to get accurate, unambigous and 
well-expressed specs. In the above case (I mean, at this level of 
detail), "accurate, unambigous and well-expressed specs" are almost 
pseudocode. Doing a bit of mind-reading (which is certainly *not* the 
right thing to do - in real life, I'd just go back to the customer or 
whoever handed me such specs to sort this out), I came out with the 
following rewrite:

1. open a given file in text mode
2. read it line by line
3. for each line:
3.1. if the line contains/startswith (?) more than three '#':
 print "an invalid line"
3.2. if the line starts with one, two or three '#':
  print as many times the line as there are '#'
3.2 else (imply : the line contains no '#'):
print "looks like a code line"

So basically, you have your algorithm. Now you just need to find out how 
to do each of these tasks in Python. That is :

a. how to open a file for reading in text mode (NB: 'text mode' may or 
not makes sense, according to the OS)

b. how to  iterate over the lines in this file once it's correctly opened

c. how to test for the presence and position of a given character / 
substring in a string

d. how to "print" something from your program.


Good news: all this is pretty well documented. I'd say that the most 
"tricky" part is c., since there's more than one possible solution, but 
since it's about strings, looking for what features Python strings has 
to offer, and trying them out in the interactive interpreter should 
solve the problem very quickly.


HTH

--
http://mail.python.org/mailman/listinfo/python-list

monitoring/restarting an application

[Ghirai]

Hello list,

I need to keep x number of instances of an external applications
running, say /bin/x, but also kill and restart each one after y seconds.

What would be the best way to do this (with python 2.5.x)?

I'm thinking of having a list of running pids, then have a thread check
if len(list) < x, and if it is, then start a new process (with
os.spawnv?) and add it to the list.
Also in the list i'd keep PIDs and some sort of started-timestamp, so i
know which to kill.

Does this sound reasonable, or is there an easier way?

Thanks.

-- 
Regards,
Ghirai.
--
http://mail.python.org/mailman/listinfo/python-list

How to access a containing objects properties from an object inside.

[nuwandame]

I have two objects obj1 and obj2. Inside obj1 there is an attribute for
success (obj1.success) and for containing other objects (obj1.data)

I am using setattr() to add obj2 as an attribute to obj1.data
(obj1.data.obj2) this is working fine.

My problem is when someone changes a variable in obj2 instance after it
is added to obj1 e.g.

obj1.data.obj2.success = False

I am trying to figure out how to locate and access obj1.success when
obj2.success has changed.


Any assistance, pointers, ideas are much appreciated.

JJ
--
http://mail.python.org/mailman/listinfo/python-list

Re: monitoring/restarting an application

[Marco Mariani]

Ghirai wrote:


I need to keep x number of instances of an external applications
running, say /bin/x, but also kill and restart each one after y seconds.

What would be the best way to do this (with python 2.5.x)?


easy_install supervisor

it should do everything for you
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to access a containing objects properties from an object inside.

[Aaron Brady]

On Mar 5, 8:44 am, nuwandame  wrote:
> I have two objects obj1 and obj2. Inside obj1 there is an attribute for
> success (obj1.success) and for containing other objects (obj1.data)
>
> I am using setattr() to add obj2 as an attribute to obj1.data
> (obj1.data.obj2) this is working fine.
>
> My problem is when someone changes a variable in obj2 instance after it
> is added to obj1 e.g.
>
> obj1.data.obj2.success = False
>
> I am trying to figure out how to locate and access obj1.success when
> obj2.success has changed.
>
> Any assistance, pointers, ideas are much appreciated.
>
> JJ

Hi.  There's no way in general, but if you will make a few
assumptions, there are some possibilities.  For example, you could
make 'obj1.success' a descriptor, which searches its '__dict__', and
looks for 'success' attributes in its contents.

How does that strike you?
--
http://mail.python.org/mailman/listinfo/python-list

Re: Number of Packages in the "cheeseshop"

[Gerard Flanagan]

Michael Rudolf wrote:
> Hi, I just wondered how many Packages are in the Python Package Index.

fwiw

http://bitbucket.org/djerdo/musette/src/tip/tools/download-pypi.py

regards

G.


--
http://mail.python.org/mailman/listinfo/python-list

RE: Parsing/Crawler Questions..

[bruce]

hi john..

You're missing the issue, so a little clarification...

I've got a number of test parsers that point to a given classlist site.. the
scripts work.

the issue that one faces is that you never "know" if you've gotten all of
the items/links that you're looking for based on the XPath functions. This
could be due to an error in the parsing, or it could be due to an admin
changing the site (removing/adding courses etc...)

So I'm trying to figure out an approach to handling these issues...

As far as I can tell... An approach might be to run the parser script across
the target site X number of times within a narrow timeframe (a few minutes).
Based on the results of this process, you might be able to develop an
overall "tree" of what the actual class/course links/list should be. But you
don't know from hour to hour, day to day if this list is stable, as it could
change..

The only way you know for certain is to physically examine a site. You can't
do this if you're going to develop an automated system for 5-10 sites, or
for 500-1000...

These are the issues that I'm grappling with.. not how to write the XPath
parsing functions...

Thanks..


-Original Message-
From: python-list-bounces+bedouglas=earthlink@python.org
[mailto:python-list-bounces+bedouglas=earthlink@python.org]on Behalf
Of John Nagle
Sent: Wednesday, March 04, 2009 10:23 PM
To: python-list@python.org
Subject: Re: Parsing/Crawler Questions..


bruce wrote:
> hi phillip...
>
> thanks for taking a sec to reply...
>
> i'm solid on the test app i've created.. but as an example.. i have a
parse
> for usc (southern cal) and it exrtacts the courselist/class schedule... my
> issue was that i realized the multiple runs of the app was giving
differentt
> results... in my case, the class schedule isn't static.. (actually, none
of
> the class/course lists need be static.. they could easily change).
>
> so i don't have apriori knowledge of what the actual class/course list
site
> would look like, unless i physically examined the site, each time i run
the
> app...
>
> i'm inclined to think i might need to run the parser a number of times
> within a given time frame, and then take a union/join of the output of the
> different runs.. this would in theory, give me a high probablity that i'd
> get 100% of the class list...

 I think I see the problem.  I took a look at the USC class list, and
it's been made "Web 2.0".  When you read the page, you don't get the
class list; you get a Javascript thing that builds a class list on
demand, using JSON, no less.

 See "http://web-app.usc.edu/soc/term_20091.html";.

 I'm not sure how you're handling this.  The Javascript actually
has to be run before you get anything.

John Nagle
--
http://mail.python.org/mailman/listinfo/python-list

--
http://mail.python.org/mailman/listinfo/python-list

Re: Roulette wheel

[mattia]

Il Thu, 05 Mar 2009 12:54:39 +0100, Peter Otten ha scritto:

> mattia wrote:
> 
>> Il Thu, 05 Mar 2009 10:46:58 +0100, Peter Otten ha scritto:
>> 
>>> mattia wrote:
>>> 
> Note how get_roulette_wheel() is now completeley independent of the
> concrete problem you are using it for.
 
 Ok, but also a lot more memory consuming ;-)
>>> 
>>> I don't think so. Python references objects; therefore the list
>>> 
>>> [tiny_little_thing]*N
>>> 
>>> does not consume more memory than
> 
> Oops, should have been less.
> 
> 
>>> [big_fat_beast]*N
>>> 
>>> Or did you have something else in mind?
>>> 
>>> Peter
>> 
>> Ok, understood. So if I have e.g. [[200 elements]]*N, then I'll have N
>> pointers to the same location of my seq, right?
> 
> Right. You can verify this with
> 
 v = [0]
 items = [v]*5
 v[0] = 42
 items
> [[42], [42], [42], [42], [42]]
> 
> which often surprises newbies.
> 
> Peter

Great explanation, thanks.
--
http://mail.python.org/mailman/listinfo/python-list

question about ctrl-d and atexit with threads

[Darren Dale]

I have a function that stops execution of a thread, and this function
is registered with atexit.register. A simple example module is
included at the end of this post, say its called test.py. If I do the
following in the interactive interpreter, the thread stops executing
as I hoped:

>>> from test import my_thread
>>> import sys
>>> sys.exit()

If instead I do the following:

>>> from test import my_thread
>>> 

the interpreter hangs up and my_thread continues to execute
indefinitely (confirmed by uncommenting the print statement in run).

I've seen this behavior on python-2.5 and 2.6 on 64 bit linux systems
(gentoo and kubuntu). Can anyone else confirm that invoking ctrl-D
hangs up the interactive interpreter with this code? And if so, could
anyone explain how ctrl-d is different than sys.exit?

Thank you,
Darren

import atexit
import threading
import time


class MyThread(threading.Thread):

def __init__(self):
threading.Thread.__init__(self)

self.lock = threading.Lock()
self.stopEvent = threading.Event()

def run(self):
while not self.stopEvent.isSet():
# print 'running'
time.sleep(0.1)

def stop(self):
self.stopEvent.set()
self.join()


my_thread = MyThread()

def stop_execution():
my_thread.stop()

atexit.register(stop_execution)

my_thread.start()
--
http://mail.python.org/mailman/listinfo/python-list

Re: Does profiling remove some optimisations?

[Gabriel Genellina]

En Thu, 05 Mar 2009 07:39:20 -0200, andrew cooke   
escribió:



I have a core loop that is critical to performance.  The code is at
http://www.acooke.org/lepl/api/lepl.parser-pysrc.html#trampoline

If I write a separate "optimised" version of that function for when
"monitor" is empty, with all the "if monitor" tests removed, the profiler
(cProfile) indicates a 10% reduction in time spent in the loop.

But if I run the same code 100 times under timeit, without profiling, I
see no difference in total time.  The process is CPU bound.


How do you measure that? The profiler is able to isolate the time spent in  
your function, and the time spent in the remaining code. But timeit isn't.  
Perhaps the 10% reduction in trampoline() is minimized against the time  
spent in the rest of the code?


--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

Re: Question about binary file reading

[vibgyorbits]

k,thanks all.

l=map(lambda x: '%02x' %ord(x),d)
s=string.join(l,sep='')

PS#. Endedup learning little bit of Lambda functions. :-)

Scott David Daniels  <<< Thanks for your wisdom about the "spaces".
Its a 3 liner code-snippet!
--
http://mail.python.org/mailman/listinfo/python-list

Re: question about ctrl-d and atexit with threads

[Darren Dale]

Actually, this problem can also be seen by running this code as a
script, it hangs up if the sys.exit lines are commented, and exits
normally if uncommented.

import atexit
import threading
import time


class MyThread(threading.Thread):

def __init__(self):
threading.Thread.__init__(self)

self.lock = threading.Lock()
self.stopEvent = threading.Event()

def run(self):
while not self.stopEvent.isSet():
time.sleep(0.1)

def stop(self):
self.stopEvent.set()
self.join()


my_thread = MyThread()

def stop_execution():
my_thread.stop()

atexit.register(stop_execution)

my_thread.start()

#import sys
#sys.exit()
--
http://mail.python.org/mailman/listinfo/python-list

Re: Question about binary file reading

[Marco Mariani]

vibgyorbits wrote:


l=map(lambda x: '%02x' %ord(x),d)
s=string.join(l,sep='')

PS#. Endedup learning little bit of Lambda functions. :-)


That's so 2007...

The 2.5-esque way to write that is

s = ''.join('%02x' % ord(x) for x in d)

--
http://mail.python.org/mailman/listinfo/python-list

Re: Question about binary file reading

[vibgyorbits]

On Mar 5, 9:24 am, Marco Mariani  wrote:
> vibgyorbits wrote:
> > l=map(lambda x: '%02x' %ord(x),d)
> > s=string.join(l,sep='')
>
> > PS#. Endedup learning little bit of Lambda functions. :-)
>
> That's so 2007...
>
> The 2.5-esque way to write that is
>
> s = ''.join('%02x' % ord(x) for x in d)

Yes..:-) I totally agree..still learning some new stuff. SOme of you
folks are really good.
thanks again.. Although, next step would be to really get into the
algorithm complexity ,but right now
let me finish my tool & then dig into finer details.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Does profiling remove some optimisations?

[andrew cooke]

Gabriel Genellina wrote:
> En Thu, 05 Mar 2009 07:39:20 -0200, andrew cooke 
> escribió:
>
>> I have a core loop that is critical to performance.  The code is at
>> http://www.acooke.org/lepl/api/lepl.parser-pysrc.html#trampoline
>>
>> If I write a separate "optimised" version of that function for when
>> "monitor" is empty, with all the "if monitor" tests removed, the
>> profiler
>> (cProfile) indicates a 10% reduction in time spent in the loop.
>>
>> But if I run the same code 100 times under timeit, without profiling, I
>> see no difference in total time.  The process is CPU bound.
>
> How do you measure that? The profiler is able to isolate the time spent in
> your function, and the time spent in the remaining code. But timeit isn't.
> Perhaps the 10% reduction in trampoline() is minimized against the time
> spent in the rest of the code?

Yeah, I wasn't very clear, and when you do the maths it's borderline in
the noise from the timeit results.

Basically the argument is something like that routine was taking 1/10 of
the total time, and I'd improved it by 10% so should see a 1% change in
total speed.  Which I thought I should be able to see (by using a large
number of iterations in timeit).

But really, worrying about a 1% total speedup is silly.  I should be
looking elsewhere.  Which is good news, I guess, because having two
versions of a core routine was a bit worrying.

Cheers,
Andrew


--
http://mail.python.org/mailman/listinfo/python-list

Re: monitoring/restarting an application

[Paul Rubin]

Ghirai  writes:
> Does this sound reasonable, or is there an easier way?

Use something like daemontools or upstart.  Don't roll your own.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Roulette wheel

[mattia]

Il Wed, 04 Mar 2009 21:30:54 +0100, Peter Otten ha scritto:

> mattia wrote:
> 
>> Hi everyone, I'm new to python and I want to create some simple code in
>> order to code the classical genetic algorithm example: given a
>> population of chromosomes, encoded using 1 and 0, find the chromosome
>> with the maximum number of 1s. Now, despite all the code used to
>> implement the solution, I'm wondering if there is a better way to use
>> the so-called roulette wheel selection in this problem. Here I paste
>> the code of my solution, any advice will be helpful:
> 
> Your code looks good to me.
> 
>> from random import randint, random
>> 
>> def create_chromosome(min, max, length):
>> chromosome = []
>> for i in range(length):
>> chromosome.append(randint(min, max))
>> return chromosome
>> 
>> def fitness(chrm, ffunc=sum):
>> return ffunc(chrm)
> 
> fitness = sum
> 
> has the same effect, without the extra indirection.
>  
>> def create_population(nelem, min, max, length):
>> return [create_chromosome(min, max, length) for i in range(nelem)]
>> 
>> def get_fitness_and_population(population):
>> return [(fitness(x), x) for x in population]
>> 
>> def get_roulette_wheel(population):
>> roulette_wheel = []
>> index = 0
>> 
>> for x in get_fitness_and_population(population):
>> for j in range(x[0]):
>> roulette_wheel.append(index)
>> index += 1
> 
> Make that
> 
>   for index, x in enumerate(get_fitness_and_population(population)):
>   ...
> 
> I'd also pass the the fitness function explicitly around instead of
> making it a global.
> 
>> return roulette_wheel
>> 
>> pop = create_population(5, 0, 1, 10)
>> rw = get_roulette_wheel(pop)
>> print(rw)
>> print(len(rw))
>> ri = randint(0, len(rw) - 1)
>> print("Random index:", rw[ri], ", value:", pop[rw[ri]])
> 
> But these are minor nits :)
> 
> Here's a slightly different approach:
> 
> from random import randint, choice
> 
> def create_chromosome(min, max, length):
> return [randint(min, max) for i in range(length)]
> 
> def create_population(nelem, min, max, length):
> return [create_chromosome(min, max, length) for i in range(nelem)]
> 
> def get_fitness_and_population(population, fitness):
> return [(fitness(x), x) for x in population]
> 
> def get_roulette_wheel(weight_value_pairs):
> roulette_wheel = []
> for weight, value in weight_value_pairs:
> roulette_wheel += [value]*weight
> return roulette_wheel
> 
> if __name__ == "__main__":
> pop = create_population(5, 0, 1, 10)
> fap = get_fitness_and_population(pop, sum) rw =
> get_roulette_wheel(fap)
> print("Random value:", choice(rw))
> 
> Note how get_roulette_wheel() is now completeley independent of the
> concrete problem you are using it for.
> 
> Peter

The last question: how can I improve readability in this piece of code?

def crossover(pop, prob=0.6):
"""
With a crossover probability cross over the parents to form new
offspring. If no crossover was performed, offspring is the exact copy of 
parents.
"""
cpop = []
for i in range(0, len(pop), 2):
# crossover
if prob > random():
crossover_point = randint(0, len(pop[i])-1)
nchromosome1 = pop[i][:crossover_point] + pop[i+1][crossover_point:]
nchromosome2 = pop[i+1][:crossover_point] + pop[i][crossover_point:]
else:
nchromosome1 = pop[i][:]
nchromosome2 = pop[i+1][:]
cpop += [nchromosome1] + [nchromosome2]
return cpop

And with this one my example is complete!
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to access a containing objects properties from an object inside.

[nuwandame]

Aaron Brady wrote:
> On Mar 5, 8:44 am, nuwandame  wrote:
>> I have two objects obj1 and obj2. Inside obj1 there is an attribute for
>> success (obj1.success) and for containing other objects (obj1.data)
>>
>> I am using setattr() to add obj2 as an attribute to obj1.data
>> (obj1.data.obj2) this is working fine.
>>
>> My problem is when someone changes a variable in obj2 instance after it
>> is added to obj1 e.g.
>>
>> obj1.data.obj2.success = False
>>
>> I am trying to figure out how to locate and access obj1.success when
>> obj2.success has changed.
>>
>> Any assistance, pointers, ideas are much appreciated.
>>
>> JJ
> 
> Hi.  There's no way in general, but if you will make a few
> assumptions, there are some possibilities.  For example, you could
> make 'obj1.success' a descriptor, which searches its '__dict__', and
> looks for 'success' attributes in its contents.
> 
> How does that strike you?

You suggest an interesting idea which triggered another idea...

Are there mechanisms for using, accessing, executing the object id?

objid = id(obj1)

If so, I could set that as an attribute in the subsequent object when
adding it as an attribute and then call it when values of that attribute
changed.
--
http://mail.python.org/mailman/listinfo/python-list

Re: how to convert from network to host byte order

[Mark Tolonen]

"Evan"  wrote in message 
news:79582a34-5d0b-49b2-8c1e-4139324ff...@b38g2000prf.googlegroups.com...

Hello ~

I'm new with python,  what my problem is, I have a binary file, I want
to read first 2 bytes and convert it to host byte order, then write it
to another file.



There is a piece of information missing here.  What is the byte order of the 
original binary file?




I try to use 'socket' and 'struct', but somehow i can not get it
working fine:

for example, totally I'm not sure if my steps is correct or not:


import socket
f=open('a.bin','rb')
f.read(2)

'\x04\x00'



This is either a little-endian 4, or a big-endian 1024 (0x400).



f.seek(0)
st=f.read(2)
e=open('test.bin','w+b')
e.write(socket.ntohs(struct.unpack('H',st[:2])[0]))

Traceback (most recent call last):
 File "", line 1, in 
TypeError: argument 1 must be string or read-only buffer, not int
+

It failed due to the parameter is 'int', not 'str' in write function.
but how can i do that?


socket.ntohs returns an integer.  write takes a string.  ntohs assumes the 
original value was big-endian (network) order.


If the original binary file is little-endian, this works:

import struct
f=open('a.bin','rb')
data = struct.unpack('e.write(struct.pack('H',data)) # default is host-order, could be big or 
little.

e.close()

If the original binary file is big-endian, change the 3rd line:

data = struct.unpack('>H',f.read(2))[0]

-Mark



--
http://mail.python.org/mailman/listinfo/python-list

Re: monitoring/restarting an application

[Ghirai]

On 05 Mar 2009 07:44:44 -0800
Paul Rubin  wrote:

> Ghirai  writes:
> > Does this sound reasonable, or is there an easier way?
> 
> Use something like daemontools or upstart.  Don't roll your own.
> --

Alright, i'll have a look at those as well as supervisor.
Thanks.

-- 
Regards,
Ghirai.
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to access a containing objects properties from an object inside.

[Aaron Brady]

On Mar 5, 9:51 am, nuwandame  wrote:
> Aaron Brady wrote:
> > On Mar 5, 8:44 am, nuwandame  wrote:
> >> I have two objects obj1 and obj2. Inside obj1 there is an attribute for
> >> success (obj1.success) and for containing other objects (obj1.data)
>
> >> I am using setattr() to add obj2 as an attribute to obj1.data
> >> (obj1.data.obj2) this is working fine.
>
> >> My problem is when someone changes a variable in obj2 instance after it
> >> is added to obj1 e.g.
>
> >> obj1.data.obj2.success = False
>
> >> I am trying to figure out how to locate and access obj1.success when
> >> obj2.success has changed.
>
> >> Any assistance, pointers, ideas are much appreciated.
>
> >> JJ
>
> > Hi.  There's no way in general, but if you will make a few
> > assumptions, there are some possibilities.  For example, you could
> > make 'obj1.success' a descriptor, which searches its '__dict__', and
> > looks for 'success' attributes in its contents.
>
> > How does that strike you?
>
> You suggest an interesting idea which triggered another idea...
>
> Are there mechanisms for using, accessing, executing the object id?
>
> objid = id(obj1)
>
> If so, I could set that as an attribute in the subsequent object when
> adding it as an attribute and then call it when values of that attribute
> changed.

I think what you are after is a weak value dictionary.  But why not
just store the parent as an attribute?  obj2.parent= obj1.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Parsing/Crawler Questions..

[John Nagle]

bruce wrote:

hi john..

You're missing the issue, so a little clarification...

I've got a number of test parsers that point to a given classlist site.. the
scripts work.

the issue that one faces is that you never "know" if you've gotten all of
the items/links that you're looking for based on the XPath functions. This
could be due to an error in the parsing, or it could be due to an admin
changing the site (removing/adding courses etc...)


   What URLs are you looking at?

John Nagle
--
http://mail.python.org/mailman/listinfo/python-list

Re: String Identity Test

[Terry Reedy]

Hendrik van Rooyen wrote:

"S Arrowsmith"  wrote:


"Small" integers get a similar treatment:


a = 256
b = 256
a is b

True

a = 257
b = 257
a is b

False


This is weird - I would have thought that the limit
of "small" would be at 255 - the biggest number to 
fit in a byte.  256 takes two bytes, so it must be

an arbitrary limit - could have been set at 300,
or 30 000...


'Small' also goes to -10 or so.  256 was included, at minuscule cost, 
because it is a relatively common number, being the number of bytes.


--
http://mail.python.org/mailman/listinfo/python-list

distutils: "build" command

[TP]

Hi everybody,

I have programmed a python package, and I would like to use distutils with
it. My package has the following structure after doing sdist and build:

$ python setup.py sdist
[...]
$ python setup.py build
[...]
$ tree
.
|-- MANIFEST
|-- MANIFEST.in
|-- README
|-- build
|   `-- lib
|   `-- utils
|   |-- __init__.py
|   `-- __init__.pyc
|-- dist
|   `-- utils-1.0.tar.gz
|-- utils
|   |-- __init__.py
|   |-- __init__.pyc
|   |-- utils_1
|   |   |-- __init__.py
|   |   |-- __init__.pyc
|   |   `-- toto.py
|   `-- utils2
|   |-- __init__.py
|   `-- foo.py
`-- setup.py

So, in the "build" directory, I do not get all the the modules ("utils1"
and "utils2" directory), but only the first level of the package (files
under "utils" directory). Whereas all the files are included in the
utils-1.0.tar.gz archive made with "sdist".
How to modify setup.py and/or MANIFEST.in to include all the subdirectories
in the "build" directory when doing "setup.py build"? I have googled a bit
without finding anything.
These are my MANIFEST.in and setup.py files:

$ cat MANIFEST.in
include README
recursive-include utils *.py

$ cat setup.py
from distutils.core import setup

setup( name = 'utils'
, version = '1.0'
, url = ''
, author = ''
, author_email = ''
, packages = [ 'utils' ] )

Thanks in advance

-- 
python -c "print ''.join([chr(154 - ord(c)) for c in '*9(9&(18%.\
9&1+,\'Z4(55l4('])"

"When a distinguished but elderly scientist states that something is
possible, he is almost certainly right. When he states that something is
impossible, he is very probably wrong." (first law of AC Clarke)
--
http://mail.python.org/mailman/listinfo/python-list

Re: distutils: "build" command

[andrew cooke]

don't know if this is useful, but setuptools is a plug-in replacement for
distutils that makes this kind of thing easier (i think).

http://peak.telecommunity.com/DevCenter/setuptools

andrew


TP wrote:
> Hi everybody,
>
> I have programmed a python package, and I would like to use distutils with
> it. My package has the following structure after doing sdist and build:
>
> $ python setup.py sdist
> [...]
> $ python setup.py build
> [...]
> $ tree
> .
> |-- MANIFEST
> |-- MANIFEST.in
> |-- README
> |-- build
> |   `-- lib
> |   `-- utils
> |   |-- __init__.py
> |   `-- __init__.pyc
> |-- dist
> |   `-- utils-1.0.tar.gz
> |-- utils
> |   |-- __init__.py
> |   |-- __init__.pyc
> |   |-- utils_1
> |   |   |-- __init__.py
> |   |   |-- __init__.pyc
> |   |   `-- toto.py
> |   `-- utils2
> |   |-- __init__.py
> |   `-- foo.py
> `-- setup.py
>
> So, in the "build" directory, I do not get all the the modules ("utils1"
> and "utils2" directory), but only the first level of the package (files
> under "utils" directory). Whereas all the files are included in the
> utils-1.0.tar.gz archive made with "sdist".
> How to modify setup.py and/or MANIFEST.in to include all the
> subdirectories
> in the "build" directory when doing "setup.py build"? I have googled a bit
> without finding anything.
> These are my MANIFEST.in and setup.py files:
>
> $ cat MANIFEST.in
> include README
> recursive-include utils *.py
>
> $ cat setup.py
> from distutils.core import setup
>
> setup( name = 'utils'
> , version = '1.0'
> , url = ''
> , author = ''
> , author_email = ''
> , packages = [ 'utils' ] )
>
> Thanks in advance
>
> --
> python -c "print ''.join([chr(154 - ord(c)) for c in '*9(9&(18%.\
> 9&1+,\'Z4(55l4('])"
>
> "When a distinguished but elderly scientist states that something is
> possible, he is almost certainly right. When he states that something is
> impossible, he is very probably wrong." (first law of AC Clarke)
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>


--
http://mail.python.org/mailman/listinfo/python-list

Re: How to access a containing objects properties from an object inside.

[nuwandame]

Aaron Brady wrote:
> On Mar 5, 9:51 am, nuwandame  wrote:
>> Aaron Brady wrote:
>>> On Mar 5, 8:44 am, nuwandame  wrote:
 I have two objects obj1 and obj2. Inside obj1 there is an attribute for
 success (obj1.success) and for containing other objects (obj1.data)
 I am using setattr() to add obj2 as an attribute to obj1.data
 (obj1.data.obj2) this is working fine.
 My problem is when someone changes a variable in obj2 instance after it
 is added to obj1 e.g.
 obj1.data.obj2.success = False
 I am trying to figure out how to locate and access obj1.success when
 obj2.success has changed.
 Any assistance, pointers, ideas are much appreciated.
 JJ
>>> Hi.  There's no way in general, but if you will make a few
>>> assumptions, there are some possibilities.  For example, you could
>>> make 'obj1.success' a descriptor, which searches its '__dict__', and
>>> looks for 'success' attributes in its contents.
>>> How does that strike you?
>> You suggest an interesting idea which triggered another idea...
>>
>> Are there mechanisms for using, accessing, executing the object id?
>>
>> objid = id(obj1)
>>
>> If so, I could set that as an attribute in the subsequent object when
>> adding it as an attribute and then call it when values of that attribute
>> changed.
> 
> I think what you are after is a weak value dictionary.  But why not
> just store the parent as an attribute?  obj2.parent= obj1.

Very cool!

Thanks a bunch.

JJ
--
http://mail.python.org/mailman/listinfo/python-list

Re: String Identity Test

[Terry Reedy]

Terry Reedy wrote:

Hendrik van Rooyen wrote:

"S Arrowsmith"  wrote:


"Small" integers get a similar treatment:


a = 256
b = 256
a is b

True

a = 257
b = 257
a is b

False


This is weird - I would have thought that the limit
of "small" would be at 255 - the biggest number to fit in a byte.  256 
takes two bytes, so it must be


Ints take as least 4 bytes.  It is commonness of usage that determined 
caching.  The range was expanded a few years ago in anticipation of the 
new bytes type, whose contents are ints, not chars.



an arbitrary limit - could have been set at 300,
or 30 000...


'Small' also goes to -10 or so.  256 was included, at minuscule cost, 
because it is a relatively common number, being the number of bytes.


In fact, 3.0.1 starts with 36 internal references to the cached int 256!

>>> import sys
>>> sys.getrefcount(256)
38 # -2 for the function call

>>> sys.getrefcount(257)
2

>>> [sys.getrefcount(i)-2 for i in range(258)]

shows that only 15 cached ints start with more references. 0 has the 
most with 724 (and that small actually goes to -5).


tjr

--
http://mail.python.org/mailman/listinfo/python-list

help understanding class or function

[Vincent Davis]

If I have a list and I what to make changes to it.a = [1,2,3,4,5,6,7,8,9]
and maybe I want to drop the odd and double the  even numbers and I will
need to do this repeatedly.
How is this best done? That is as a function or class. All I know how to do
is

def doubleeven(alist):
blist = [2*x for x in a if x % 2 ==0]
   return blist

then when I am using it I do this

a = doubleeven(a)
I what to keep it named "a"

I am not sure if this is the best way in terms of format or structure. Is
there a better way.

Thanks
Vincent Davis
720-301-3003
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pickle Problem

[Fab86]

On Mar 4, 2:49 pm, MRAB  wrote:
> Fab86 wrote:
> > On Mar 4, 1:40 am, "Gabriel Genellina"  wrote:
> >> En Tue, 03 Mar 2009 23:11:30 -0200, Fab86  escribió:
>
> >>> On Mar 4, 12:00 am, MRAB  wrote:
>  Fab86 wrote:
> > On Mar 3, 8:59 pm, "Gabriel Genellina"  wrote:
> >> How to "spell" exactly the exception name should appear in the  
> >> documentation; might be yahoo.SearchError, or  
>  yahoo.search.SearchError, or  
> >> yahoo.errors.SearchError, or similar.
> > I have been trying except SearchError: however I get the error:
> > Traceback (most recent call last):
> >   File "C:\Downloads\MoS\yws-2.12\Python\pYsearch-3.1\timeDelay.py",
> > line 19, in 
> >     except SearchError:
> > NameError: name 'SearchError' is not defined
> > I have searched all documents for terms along the lines of searcherror
> > but am finding nothing..
>  It's defined in the module you imported to get the search functionality.
> >>> I imported:
> >>> from yahoo.search.web import WebSearch
> >>> However there is nothing re SearchError in that doc or in the .py.
> >>> I can only find a reference to SearchError in the __init__ file as a
> >>> class called SearchError
> >> The __init__.py indicates a package  
> >> 
> >> You didn't tell the package name (the name of the directory containing  
> >> __init__.py) so this is somewhat generic. If the package name is foo, use:
> >>  from foo import SearchError
> >> If foo is a subpackage under bar, use:
> >>  from bar.foo import SearchError
>
> >> It *might* be:
> >>  from yahoo.search.web import WebSearch
> >> or perhaps:
> >>  from yahoo.search import WebSearch
>
> >> You can enter those lines in the interactive interpreter to discover the  
> >> right form. (This really ought to have been documented)
>
> >> --
> >> Gabriel Genellina
>
> > Ok, I managed to import the correct error class (was in a non expected
> > place)
>
> > Just as I thought I was finished, I encountered a final problem. I am
> > running a while loop which is constantly adding search results to a
> > file, like this (print >> f, res.total_results_available). I have put
> > an exception in that if it times out, it simply tries again until its
> > finished. The problem is that the loop re-write all new results
> > continuing on from the previous searches. I would like to somehow
> > delete all in that file and start again.
>
> > I thought this could simply be achieved by putting f.close() in the
> > exception and then it re-writes it however I am getting this error:
>
> > Traceback (most recent call last):
> >   File "/home/csunix/scs5fjnh/FYProj/Python/pYsearch-3.1/test9.py",
> > line 17, in 
> >     print >> f, res.total_results_available
> > ValueError: I/O operation on closed file
>
> > Is there another way rather than closing the file? Is it possible to
> > delete all within the file?
>
> You could do:
>
>      f.seek(0)
>      f.truncate()
>
> I hope you don't just discard all the results you've got so far and then
> start from the beginning again.

MRAB, yes I do, thats the only way I can I get it to work. I wouldnt
know how to make the program read  the expection and then carry of
from the last search term.

Is that even possible?
--
http://mail.python.org/mailman/listinfo/python-list

Re: Inverse of dict(zip(x,y))

[Tino Wildenhain]

Piet van Oostrum wrote:

Andre Engels  (AE) wrote:



AE> On Wed, Mar 4, 2009 at 11:02 AM, lone_eagle  wrote:

Can someone suggest a easy method to do the inverse of dict(zip(x,y))
to get two lists x and y?

So, if x and y are two lists, it is easier to make a dictionary using
d = dict(zip(x,y)), but if I have d of the form, d = {x1:y1,
x2:y2, ...}, what is there any trick to get lists x = [x1, x2, ...]
and y = [y1, y2, ...]



AE> x = d.keys()
AE> y = [d[e] for d in x]



AE> y = d.values() might also work, but I am not sure whether d.keys() and
AE> d.values() are guaranteed to use the same order.


Yes, they are if the dictionary is not changed in the meantime (not even
inserting and removing the same thing). See the library documentation,
section dict.


Still I'd like to see an application where this really matters (that
keys() and values() match in order)

Tino


smime.p7s
Description: S/MIME Cryptographic Signature
--
http://mail.python.org/mailman/listinfo/python-list

Re: question about ctrl-d and atexit with threads

[skip]

What happens if you simply call

my_thread.setDaemon(True)

(or in Python 2.6):

my_thread.daemon = True

?  That is the documented way to exit worker threads when you want the
application to exit.  From the threading module docs:

"The entire Python program exits when no alive non-daemon threads are
left."

Skip
--
http://mail.python.org/mailman/listinfo/python-list

While loop

[Fab86]

Hello,

I am currently working on my program which send queries to Yahoo and
then saves them into a flatfile. The problem I have is that I need to
conduct 200 searches and Yahoo typically times out during the search
with an error. I have caught the error and told it to time.sleep(10)
however I can only think of deleting the flatfile and starting again
hoping that it will not time out again.

Is it possible to get the program to catch the exception, wait 10
seconds, then carry of from where it was rather than starting again?

If so, could someone please inform me how to do this?

Thanks,

Fabien
--
http://mail.python.org/mailman/listinfo/python-list

Re: Roulette wheel

[Peter Otten]

mattia wrote:

> The last question: how can I improve readability in this piece of code?
> 
> def crossover(pop, prob=0.6):
> """
> With a crossover probability cross over the parents to form new
> offspring. If no crossover was performed, offspring is the exact copy
> of parents. """
> cpop = []
> for i in range(0, len(pop), 2):
> # crossover
> if prob > random():
> crossover_point = randint(0, len(pop[i])-1)
> nchromosome1 = pop[i][:crossover_point] +
> pop[i+1][crossover_point:] nchromosome2 =
> pop[i+1][:crossover_point] + pop[i][crossover_point:]
> else:
> nchromosome1 = pop[i][:]
> nchromosome2 = pop[i+1][:]

> cpop += [nchromosome1] + [nchromosome2]

I'd write that as 

cpop.append(nchromosome1)
cpop.append(nchromosome2)

thus avoiding the intermediate lists.

> return cpop
> 
> And with this one my example is complete!

Just for fun here's an alternative version of your function

def generate_crossover(pop, prob):
for a, b in zip(*[iter(pop)]*2):
if prob > random():
cut = randrange(len(a))
a, b = a[:cut] + b[cut:], b[:cut] + a[cut:]
yield a
yield b

def crossover(pop, prob=0.6):
return list(generate_crossover(pop, prob))

but as the original is a bit clearer I recommend that you stick with it.

Peter
--
http://mail.python.org/mailman/listinfo/python-list

Python3 on the Web

[Johannes Permoser]

Hi,

I wanted to learn Python from scratch and start off with Version 3.
Since I already know PHP very well, I thought it would be nice to start
off with a small web-project.

But what's the way to bring python3 to the Web?
mod_python isn't available, cgi is said to be slow, mod_wsgi looks
complicated...

What would you suggest?

Greets, JP
--
http://mail.python.org/mailman/listinfo/python-list

Re: Help required to read and print lines based on the type of first character

[Bruno Desthuilliers]

abhinayaraj.r...@emulex.com a écrit :


Please, don't top-post, and learn to quote & snip
(if you don't know what top-posting is, google is your friend).



Thank you so much for your guidance, Bruno.

This should help me in a long way.

Here is the code I have written.

path = raw_input("\nEnter location eg. c:/buffer/test.txt : \n") 
fileIN = open(path)


This will break if the file can't be opened (doesn't exist, is 
protected, whatever).



count = 0


'count' is used locally in the loop, so it shouldn't be set outside it.


for line in fileIN:
data= line

if '' in data:
count = 4
elif '###' in data:
count = 3
elif '##' in data:
count = 2
elif '#' in data:
count = 1
elif data.find('#') == -1:


This test is redundant. if the "'#' in data" evals to False, then 
data.find('#') is garanteed to return -1.



count = 0


if (count == 0 and data!=""):


the file iterator yields lines with the newline character included, so 
there's no way that data == "" (unless you explicitely remove the 
newline character yourself). Read doc for the strip method of string 
objects.


Also, you don't need the parens.


print data + '\nlooks like a code line...\n'
elif(count== 4):
print data + '\ninvalid line!\n'
elif count>=1:
for i in range(0, count):
print data


You didn't address my questions wrt/ specs clarifications. There's a 
*big* difference between *containing* a substring and *starting with* a 
substring. Hint: Python strings have a "startswith" method...


Also and FWIW, since Python >= 2.5.2 (and possibly >= 2.5.0 - I let you 
check the docs), Python's strings have a count method too.


Your code is not too bad for a beginner - I've done worse when I started 
-, but it doesn't really matches the specs (which is impossible FWIW 
given the ambiguities they contain, cf my previous post), and contains a 
couple of more or less useless or redundant tests which make it looks a 
bit like "programming by accident", and doesn't really uses Python's 
string handling features.


HTH

--
http://mail.python.org/mailman/listinfo/python-list

Re: While loop

[Tino Wildenhain]

Hi,

Fab86 wrote:

Hello,

I am currently working on my program which send queries to Yahoo and
then saves them into a flatfile. The problem I have is that I need to
conduct 200 searches and Yahoo typically times out during the search
with an error. I have caught the error and told it to time.sleep(10)
however I can only think of deleting the flatfile and starting again
hoping that it will not time out again.

Is it possible to get the program to catch the exception, wait 10
seconds, then carry of from where it was rather than starting again?

If so, could someone please inform me how to do this?


for search in range(200):
for t in range(MAXRETRY):
try:
result=requestfromserver(...)
break
except IOError,e: # or whaterver error you get
time.sleep(10)
# log something ...
else: # if above runs w/o break
raise XYZError...() # define appropriate for your app

# here you can work with result


HTH
Tino


smime.p7s
Description: S/MIME Cryptographic Signature
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python3 on the Web

[Jean-Paul Calderone]

On Thu, 05 Mar 2009 18:13:05 +0100, Johannes Permoser  wrote:

Hi,

I wanted to learn Python from scratch and start off with Version 3.
Since I already know PHP very well, I thought it would be nice to start
off with a small web-project.

But what's the way to bring python3 to the Web?
mod_python isn't available, cgi is said to be slow, mod_wsgi looks
complicated...

What would you suggest?


Try Python 2.x instead.  If you go with Python 3, you're going to run
into the same limited library selection over and over.

Jean-Paul
--
http://mail.python.org/mailman/listinfo/python-list

Sphinx for API documentation

[Paul Hildebrandt]

I really like the look of Sphinx BUT I want autogenerated
documentation like Epydoc/doxygen.  Does anyone know of a frontend for
Sphix that will make it work like Epydoc?

Paul
--
http://mail.python.org/mailman/listinfo/python-list

Re: While loop

[Marco Mariani]

Fab86 wrote:


Is it possible to get the program to catch the exception, wait 10
seconds, then carry of from where it was rather than starting again?



something like this? probably works in PASCAL as well :)


i=0
while i < len(stuff):
   try:
  do_with(stuff[i])
   except SomeError:
  sleep(10)
  continue
   i+=1


--
http://mail.python.org/mailman/listinfo/python-list

Re: Python3 on the Web

[Bruno Desthuilliers]

Johannes Permoser a écrit :

Hi,

I wanted to learn Python from scratch and start off with Version 3.
Since I already know PHP very well, I thought it would be nice to start
off with a small web-project.

But what's the way to bring python3 to the Web?
mod_python isn't available, cgi is said to be slow,


This comes from the CGI architecture by itself (IOW: it's totally 
unrelated to the language used).



mod_wsgi looks
complicated...


It isn't, really - at least if you have enough knowledge of the HTTP 
protocol, webservers and Python !-)



What would you suggest?



If what you want is to learn *Python*, start with CGI (it'll teach you a 
lot about text processing and even possibily about web programming too) 
- but be prepared to find CGI *very* low-level compared to PHP.


Else, forget about Python 3, and try some of the nice frameworks like 
Django, Pylons, web.py etc...


--
http://mail.python.org/mailman/listinfo/python-list

Re: question about ctrl-d and atexit with threads

[Darren Dale]

On Mar 5, 12:02 pm, s...@pobox.com wrote:
> What happens if you simply call
>
>     my_thread.setDaemon(True)
>
> (or in Python 2.6):
>
>     my_thread.daemon = True
>
> ?  That is the documented way to exit worker threads when you want the
> application to exit.  From the threading module docs:
>
>     "The entire Python program exits when no alive non-daemon threads are
>     left."

Thank you Skip, that solves the problem. I'm still curious what the
difference is between python's handling of sys.exit and EOF, but its
academic at this point.

Thanks again,
Darren
--
http://mail.python.org/mailman/listinfo/python-list

RE: Parsing/Crawler Questions..

[bruce]

hi..

the url i'm focusing on is irrelevant to the issue i'm trying to solve at
this time.

i think an approach will be to fire up a number of parsing attempts, and to
track the returned depts/classes/etc... in theory (hopefully) i should be
able to create a process to build a kind of statistical representation of
what the site looks like (names of depts, names/number of classes for given
depts, etc..) if i'm correct, this would provide a complete
"list/understanding" of what the courselist looks like

i could then run the parsing process a number of times, examining the actual
value/results for the query, and taking the highest/oldest values for the
given query.. the idea being that the app will return correct results for
most of the queries, most of the time.. so from a statistical basis.. i can
take the results that are returned with the highest frequency...

so this approach might work. but again, haven't seen anything in the
literature/'net that talks about this...


thoughts...

thanks



-Original Message-
From: python-list-bounces+bedouglas=earthlink@python.org
[mailto:python-list-bounces+bedouglas=earthlink@python.org]on Behalf
Of John Nagle
Sent: Thursday, March 05, 2009 8:38 AM
To: python-list@python.org
Subject: Re: Parsing/Crawler Questions..


bruce wrote:
> hi john..
>
> You're missing the issue, so a little clarification...
>
> I've got a number of test parsers that point to a given classlist site..
the
> scripts work.
>
> the issue that one faces is that you never "know" if you've gotten all of
> the items/links that you're looking for based on the XPath functions. This
> could be due to an error in the parsing, or it could be due to an admin
> changing the site (removing/adding courses etc...)

What URLs are you looking at?

John Nagle
--
http://mail.python.org/mailman/listinfo/python-list

--
http://mail.python.org/mailman/listinfo/python-list

Re: Parsing/Crawler Questions..

[Philip Semanchuk]

On Mar 5, 2009, at 12:31 PM, bruce wrote:


hi..

the url i'm focusing on is irrelevant to the issue i'm trying to  
solve at

this time.


Not if we're to understand the situation you're trying to describe.  
From what I can tell, you're saying that the target site displays  
different results each time your crawler visits it. It's as if e.g.  
the site knows about 100 courses but only displays 80 randomly chosen  
ones to each visitor. If that's the case, then it is truly bizarre.










i think an approach will be to fire up a number of parsing attempts,  
and to
track the returned depts/classes/etc... in theory (hopefully) i  
should be
able to create a process to build a kind of statistical  
representation of
what the site looks like (names of depts, names/number of classes  
for given

depts, etc..) if i'm correct, this would provide a complete
"list/understanding" of what the courselist looks like

i could then run the parsing process a number of times, examining  
the actual
value/results for the query, and taking the highest/oldest values  
for the
given query.. the idea being that the app will return correct  
results for
most of the queries, most of the time.. so from a statistical  
basis.. i can

take the results that are returned with the highest frequency...

so this approach might work. but again, haven't seen anything in the
literature/'net that talks about this...


thoughts...

thanks



-Original Message-
From: python-list-bounces+bedouglas=earthlink@python.org
[mailto:python-list-bounces+bedouglas=earthlink@python.org]on  
Behalf

Of John Nagle
Sent: Thursday, March 05, 2009 8:38 AM
To: python-list@python.org
Subject: Re: Parsing/Crawler Questions..


bruce wrote:

hi john..

You're missing the issue, so a little clarification...

I've got a number of test parsers that point to a given classlist  
site..

the

scripts work.

the issue that one faces is that you never "know" if you've gotten  
all of
the items/links that you're looking for based on the XPath  
functions. This
could be due to an error in the parsing, or it could be due to an  
admin

changing the site (removing/adding courses etc...)


   What URLs are you looking at?

John Nagle
--
http://mail.python.org/mailman/listinfo/python-list

--
http://mail.python.org/mailman/listinfo/python-list


--
http://mail.python.org/mailman/listinfo/python-list

Re: Inverse of dict(zip(x,y))

[Andre Engels]

On Thu, Mar 5, 2009 at 6:01 PM, Tino Wildenhain  wrote:

> Still I'd like to see an application where this really matters (that
> keys() and values() match in order)

I think there are many such applications, but also that in each of
those cases it's a mis-programming of something that would be done
better and more pythonic using items()...


-- 
André Engels, andreeng...@gmail.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: While loop

[Fab86]

On Mar 5, 5:23 pm, Marco Mariani  wrote:
> Fab86 wrote:
> > Is it possible to get the program to catch the exception, wait 10
> > seconds, then carry of from where it was rather than starting again?
>
> something like this? probably works in PASCAL as well :)
>
> > i=0
> > while i < len(stuff):
> >    try:
> >       do_with(stuff[i])
> >    except SomeError:
> >       sleep(10)
> >       continue
> >    i+=1
>
>

using sleep and then continue just makes the search start from the
first search term like before.. Would it be easier to understand if I
posted sections of my code?

i = 0
k = open('blah', 'w')
domains = ["au", "ca", "nl", "be", "...]
srch = WebSearch(app_id=YahooKey)

while i<200:
try:
for domain in domains:
srch.query = "test site:.%s" % domain
res = srch.parse_results()
print >> k, res.total_results_available
i = i + 1

except SearchError:

(I currently close then reopen document here then restart i to 0)

Any ideas?
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pickle Problem

[MRAB]

Fab86 wrote:

On Mar 4, 2:49 pm, MRAB  wrote:

Fab86 wrote:

On Mar 4, 1:40 am, "Gabriel Genellina"  wrote:

En Tue, 03 Mar 2009 23:11:30 -0200, Fab86  escribió:

On Mar 4, 12:00 am, MRAB  wrote:

Fab86 wrote:

On Mar 3, 8:59 pm, "Gabriel Genellina"  wrote:
How to "spell" exactly the exception name should appear in the  
documentation; might be yahoo.SearchError, or  
yahoo.search.SearchError, or  

yahoo.errors.SearchError, or similar.

I have been trying except SearchError: however I get the error:
Traceback (most recent call last):
  File "C:\Downloads\MoS\yws-2.12\Python\pYsearch-3.1\timeDelay.py",
line 19, in 
except SearchError:
NameError: name 'SearchError' is not defined
I have searched all documents for terms along the lines of searcherror
but am finding nothing..

It's defined in the module you imported to get the search functionality.

I imported:
from yahoo.search.web import WebSearch
However there is nothing re SearchError in that doc or in the .py.
I can only find a reference to SearchError in the __init__ file as a
class called SearchError
The __init__.py indicates a package  

You didn't tell the package name (the name of the directory containing  
__init__.py) so this is somewhat generic. If the package name is foo, use:

 from foo import SearchError
If foo is a subpackage under bar, use:
 from bar.foo import SearchError
It *might* be:
 from yahoo.search.web import WebSearch
or perhaps:
 from yahoo.search import WebSearch
You can enter those lines in the interactive interpreter to discover the  
right form. (This really ought to have been documented)

--
Gabriel Genellina

Ok, I managed to import the correct error class (was in a non expected
place)
Just as I thought I was finished, I encountered a final problem. I am
running a while loop which is constantly adding search results to a
file, like this (print >> f, res.total_results_available). I have put
an exception in that if it times out, it simply tries again until its
finished. The problem is that the loop re-write all new results
continuing on from the previous searches. I would like to somehow
delete all in that file and start again.
I thought this could simply be achieved by putting f.close() in the
exception and then it re-writes it however I am getting this error:
Traceback (most recent call last):
  File "/home/csunix/scs5fjnh/FYProj/Python/pYsearch-3.1/test9.py",
line 17, in 
print >> f, res.total_results_available
ValueError: I/O operation on closed file
Is there another way rather than closing the file? Is it possible to
delete all within the file?

You could do:

 f.seek(0)
 f.truncate()

I hope you don't just discard all the results you've got so far and then
start from the beginning again.


MRAB, yes I do, thats the only way I can I get it to work. I wouldnt
know how to make the program read  the expection and then carry of
from the last search term.

Is that even possible?


Your code looks something like this:

srch = WebSearch(app_id=YahooKey)
for lang in langs:
srch.query = "avoir site:.%s" % lang
res = srch.parse_results()
print >> f, res.total_results_available

The exception is raise by, I think, srch.parse_results(), so you could
do something like this:

srch = WebSearch(app_id=YahooKey)
for lang in langs:
srch.query = "avoir site:.%s" % lang
# Use a loop in case we need to try again.
while True:
   try:
   res = srch.parse_results()
   except SearchError:
   # Failed. Wait a while before trying again.
   time.sleep(SLEEP_IN_SECONDS)
   else:
   # No exception, so we have the results.
   break
print >> f, res.total_results_available

I'm assuming that you can re-use the WebSearch instance for repeated 
queries even if it raised the SearchError exception.


If not, just put another srch = WebSearch(app_id=YahooKey) line after
the sleep to create a new WebSearch instance.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Roulette wheel

[Scott David Daniels]

mattia wrote:

... The last question: how can I improve readability in this piece of code?

def crossover(pop, prob=0.6):
"""
With a crossover probability cross over the parents to form new
offspring. If no crossover was performed, offspring is the exact copy of 
parents.
"""
cpop = []
for i in range(0, len(pop), 2):
# crossover
if prob > random():
crossover_point = randint(0, len(pop[i])-1)
nchromosome1 = pop[i][:crossover_point] + pop[i+1][crossover_point:]
nchromosome2 = pop[i+1][:crossover_point] + pop[i][crossover_point:]
else:
nchromosome1 = pop[i][:]
nchromosome2 = pop[i+1][:]
cpop += [nchromosome1] + [nchromosome2]
return cpop


Python encourages iterating over data, not indices.
Just for fun, I'll do it as a generator:
def crossover(pop, prob=0.6):
'''
note the population is treated pairwise, so any individual
is only likely to cross with one particular other individual.
'''
for first, second in zip(pop[::2], pop[1::2]):
if prob > random():
spot = randint(1, len(first) - 1) # avoid non-crossings
yield first[: spot] + second[spot :]
yield second[: spot] + first[spot :]
else:
# consider not even producing the duplicates
yield first
yield second


def random_crossing(population, probability=0.6):
'''This chooses random pairs from the population.
'''
shuffled = list(population) # make a copy
random.shuffle(shuffled) # mix it up
return list(crossover(shuffled)) # convert back to a list.


--Scott David Daniels
scott.dani...@acm.org
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python3 on the Web

[Christian Heimes]

Johannes Permoser wrote:
> But what's the way to bring python3 to the Web?
> mod_python isn't available, cgi is said to be slow, mod_wsgi looks
> complicated...

WSGI is the right way to go. It's *the* Python standard for web
application interfaces. You don't need mod_wsgi and Apache to run a WSGI
based application. In facts it's much easier to use some of the smaller
WSGI servers for development. I recommend against mod_python for a new
project.

CherryPy (http://www.cherrypy.org) might be interesting for you. It lets
you write a small application in a few minutes but it's powerful enough
to serve large applications, too.

Viel Erfolg!
Christian

--
http://mail.python.org/mailman/listinfo/python-list

Re: Inverse of dict(zip(x,y))

[psykeedelik]

On Mar 5, 6:01 pm, Tino Wildenhain  wrote:
> Piet van Oostrum wrote:
> >> Andre Engels  (AE) wrote:
>
> >> AE> On Wed, Mar 4, 2009 at 11:02 AM, lone_eagle  wrote:
>  Can someone suggest a easy method to do the inverse of dict(zip(x,y))
>  to get two lists x and y?
>
>  So, if x and y are two lists, it is easier to make a dictionary using
>  d = dict(zip(x,y)), but if I have d of the form, d = {x1:y1,
>  x2:y2, ...}, what is there any trick to get lists x = [x1, x2, ...]
>  and y = [y1, y2, ...]
>
> >> AE> x = d.keys()
> >> AE> y = [d[e] for d in x]
>
> >> AE> y = d.values() might also work, but I am not sure whether d.keys() and
> >> AE> d.values() are guaranteed to use the same order.
>
> > Yes, they are if the dictionary is not changed in the meantime (not even
> > inserting and removing the same thing). See the library documentation,
> > section dict.
>
> Still I'd like to see an application where this really matters (that
> keys() and values() match in order)
>
> Tino
>
>  smime.p7s
> 4KViewDownload

First, thanks to all the guys who posted replies to my query!!

And then, just as an example to what Tino raised ...

I usually get properties that I compute, in a dictionary like property
= [key1: val1, key2:val2, ...] and then I usually want to plot them in
pylab, which AFAIK requires x and y as lists for the plot argument.
Then I need to get the lists [key1, key2, ...] and [val1, val2, ...].
And now I wonder if there a more efficient way doing what I described
above!! ;)

Cheers,
Chaitanya
--
http://mail.python.org/mailman/listinfo/python-list

Re: Parsing/Crawler Questions - solution

[John Nagle]

Philip Semanchuk wrote:

On Mar 5, 2009, at 12:31 PM, bruce wrote:


hi..

the url i'm focusing on is irrelevant to the issue i'm trying to solve at
this time.


Not if we're to understand the situation you're trying to describe. From 
what I can tell, you're saying that the target site displays different 
results each time your crawler visits it. It's as if e.g. the site knows 
about 100 courses but only displays 80 randomly chosen ones to each 
visitor. If that's the case, then it is truly bizarre.


Agreed.  The course list isn't changing that rapidly.

I suspect the original poster is doing something like reading the DOM
of a dynamic page while the page is still updating, running a browser
in a subprocess.  Is that right?

I've had to deal with that in Javascript.  My AdRater browser plug-in
(http://www.sitetruth.com/downloads) looks at Google-served ads and
rates the advertisers.   There, I have to watch for page-change events
and update the annotations I'm adding to ads.

But you don't need to work that hard here. The USC site is actually
querying a server which provides the requested data in JSON format.  See

http://web-app.usc.edu/soc/dev/scripts/soc.js

Reverse-engineer that and you'll be able to get the underlying data.
(It's an amusing script; many little fixes to data items are performed,
something that should have been done at the database front end.)

The way to get USC class data is this:

1.  Start here: "http://web-app.usc.edu/soc/term_20091.html";
2.  Examine all the department pages under that page.
3.  On each page, look for the value of "coursesrc", like this:
var coursesrc = '/ws/soc/api/classes/aest/20091'
4.  For each "coursesrc" value found, construct a URL like this:
http://web-app.usc.edu/ws/soc/api/classes/aest/20091
5.  Read that URL.  This will return the department's course list in
JSON format.
6.  From the JSON tree, pull out CourseData items, which look like this:

CourseData":
{"prefix":"AEST",
"number":"220",
"sequence":"B",
"suffix":{},
"title":"Advanced Leadership Laboratory II",
"description":"Additional exposure to the military experience for continuing 
AFROTC cadets, focusing on customs and courtesies, drill and ceremonies, and the 
environment of an Air Force officer. Credit\/No Credit.",

"units":"1",
"restriction_by_major":{},
"restriction_by_class":{},
"restriction_by_school":{},
"CourseNotes":{},
"CourseTermNotes":{},
"prereq_text":"AEST-220A",
"coreq_text":{},
"SectionData":{"id":"41799",
"session":"790",
"dclass_code":"D",
"title":"Advanced Leadership Laboratory II",
"section_title":{},
"description":{},
"notes":{},
"type":"Lec",
"units":"1",
"spaces_available":"30",
"number_registered":"2",
"wait_qty":"0",
"canceled":"N",
"blackboard":"Y",
"comment":{},
"day":{},"start_time":"TBA",
"end_time":"TBA",
"location":"OFFICE",
"instructor":{"last_name":"Hampton","first_name":"Daniel"},
"syllabus":{"format":{},"filesize":{}},
"IsDistanceLearning":"N"}}},

Parsing the JSON is left as an exercise for the student.  (There's
a Python module for that.)

And no, the data isn't changing; you can read those pages of JSON over and
over and get the same data every time.

John Nagle
--
http://mail.python.org/mailman/listinfo/python-list

Re: Inverse of dict(zip(x,y))

[Andre Engels]

On Thu, Mar 5, 2009 at 7:07 PM, psykeedelik  wrote:

> I usually get properties that I compute, in a dictionary like property
> = [key1: val1, key2:val2, ...] and then I usually want to plot them in
> pylab, which AFAIK requires x and y as lists for the plot argument.
> Then I need to get the lists [key1, key2, ...] and [val1, val2, ...].
> And now I wonder if there a more efficient way doing what I described
> above!! ;)

If the dict = {key1: val1, key2: val2, ...}, you can do:

for key in dict:
plot(key,dictionary[key])



-- 
André Engels, andreeng...@gmail.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: Inverse of dict(zip(x,y))

[Andre Engels]

On Thu, Mar 5, 2009 at 7:46 PM, Andre Engels  wrote:

> If the dict = {key1: val1, key2: val2, ...}, you can do:
>
> for key in dict:
>    plot(key,dictionary[key])

Of course I meant:

for key in dict:
   plot(key,dict[key])



-- 
André Engels, andreeng...@gmail.com
--
http://mail.python.org/mailman/listinfo/python-list

RE: Parsing/Crawler Questions - solution

[bruce]

john...

again the problem i'm facing really has nothing to do with a specific
url... the app i have for the usc site works...

but for any number of reasons... you might get different results when
running the app..
-the server could be screwed up..
-data might be cached
-data might be changed, and not updated..
-actual app problems...
-networking issues...
-memory corruption issues...
-process constraint issues..
-web server overload..
-etc...

the assumption that most people appear to make is that if you create a
parser, and run and test it once.. then if it gets you the data, it's
working.. when you run the same app.. 100s of times, and you're slamming the
webserver... then you realize that that's a vastly different animal than
simply running a snigle query a few times...

so.. nope, i'm not running the app and getting data from a dynamic page that
hasn't finished uploading/creating the content..

but what my analysis is showing, not only for the usc, but for others as
well.. is that there might be differences in what gets returned...

which is where a smoothing algorithmic approach appears to be workable..

i've been starting to test this approach, and it actually might have a
chance of working...

so.. as i've stated a number of times.. focusing on a specific url isn't the
issue.. the larger issue is how you can
programatically/algorithmically/automatically, be reasonably ensured that
what you have is exactly what's on the site...

ain't screen scraping fun!!!



-Original Message-
From: python-list-bounces+bedouglas=earthlink@python.org
[mailto:python-list-bounces+bedouglas=earthlink@python.org]on Behalf
Of John Nagle
Sent: Thursday, March 05, 2009 10:54 AM
To: python-list@python.org
Subject: Re: Parsing/Crawler Questions - solution


Philip Semanchuk wrote:
> On Mar 5, 2009, at 12:31 PM, bruce wrote:
>
>> hi..
>>
>> the url i'm focusing on is irrelevant to the issue i'm trying to solve at
>> this time.
>
> Not if we're to understand the situation you're trying to describe. From
> what I can tell, you're saying that the target site displays different
> results each time your crawler visits it. It's as if e.g. the site knows
> about 100 courses but only displays 80 randomly chosen ones to each
> visitor. If that's the case, then it is truly bizarre.

 Agreed.  The course list isn't changing that rapidly.

 I suspect the original poster is doing something like reading the DOM
of a dynamic page while the page is still updating, running a browser
in a subprocess.  Is that right?

 I've had to deal with that in Javascript.  My AdRater browser plug-in
(http://www.sitetruth.com/downloads) looks at Google-served ads and
rates the advertisers.   There, I have to watch for page-change events
and update the annotations I'm adding to ads.

 But you don't need to work that hard here. The USC site is actually
querying a server which provides the requested data in JSON format.  See

http://web-app.usc.edu/soc/dev/scripts/soc.js

Reverse-engineer that and you'll be able to get the underlying data.
(It's an amusing script; many little fixes to data items are performed,
something that should have been done at the database front end.)

The way to get USC class data is this:

1.  Start here: "http://web-app.usc.edu/soc/term_20091.html";
2.  Examine all the department pages under that page.
3.  On each page, look for the value of "coursesrc", like this:
var coursesrc = '/ws/soc/api/classes/aest/20091'
4.  For each "coursesrc" value found, construct a URL like this:
http://web-app.usc.edu/ws/soc/api/classes/aest/20091
5.  Read that URL.  This will return the department's course list in
 JSON format.
6.  From the JSON tree, pull out CourseData items, which look like this:

CourseData":
{"prefix":"AEST",
"number":"220",
"sequence":"B",
"suffix":{},
"title":"Advanced Leadership Laboratory II",
"description":"Additional exposure to the military experience for continuing
AFROTC cadets, focusing on customs and courtesies, drill and ceremonies, and
the
environment of an Air Force officer. Credit\/No Credit.",
"units":"1",
"restriction_by_major":{},
"restriction_by_class":{},
"restriction_by_school":{},
"CourseNotes":{},
"CourseTermNotes":{},
"prereq_text":"AEST-220A",
"coreq_text":{},
"SectionData":{"id":"41799",
"session":"790",
"dclass_code":"D",
"title":"Advanced Leadership Laboratory II",
"section_title":{},
"description":{},
"notes":{},
"type":"Lec",
"units":"1",
"spaces_available":"30",
"number_registered":"2",
"wait_qty":"0",
"canceled":"N",
"blackboard":"Y",
"comment":{},
"day":{},"start_time":"TBA",
"end_time":"TBA",
"location":"OFFICE",
"instructor":{"last_name":"Hampton","first_name":"Daniel"},
"syllabus":{"format":{},"filesize":{}},
"IsDistanceLearning":"N"}}},

Parsing the JSON is left as an exercise for the student.  (There's
a Python module for that.)

And no, the data isn't changing; you can read those pages of JSON over and
over and get the same dat

Re: While loop

[Andre Engels]

On Thu, Mar 5, 2009 at 6:49 PM, Fab86  wrote:
> On Mar 5, 5:23 pm, Marco Mariani  wrote:
>> Fab86 wrote:
>> > Is it possible to get the program to catch the exception, wait 10
>> > seconds, then carry of from where it was rather than starting again?
>>
>> something like this? probably works in PASCAL as well :)
>>
>> > i=0
>> > while i < len(stuff):
>> >    try:
>> >       do_with(stuff[i])
>> >    except SomeError:
>> >       sleep(10)
>> >       continue
>> >    i+=1
>>
>>
>
> using sleep and then continue just makes the search start from the
> first search term like before.. Would it be easier to understand if I
> posted sections of my code?
>
> i = 0
> k = open('blah', 'w')
> domains = ["au", "ca", "nl", "be", "...]
> srch = WebSearch(app_id=YahooKey)
>
> while i<200:
>    try:
>        for domain in domains:
>            srch.query = "test site:.%s" % domain
>            res = srch.parse_results()
>            print >> k, res.total_results_available
>            i = i + 1
>
>    except SearchError:
>
> (I currently close then reopen document here then restart i to 0)
>
> Any ideas?

What you did not tell us was the inside loop. Just put code like
Tino's inside this inner loop:


while i<200:
   for domain in domains:
   srch.query = "test site:.%s" % domain
   while true:
   try:
res = srch.parse_results()
break
   except SearchError:
time.sleep(10)
   print >> k, res.total_results_available
   i = i + 1


Or, if you want to to retry a maximum number of times:


max_retries = 50
while i<200:
   for domain in domains:
   srch.query = "test site:.%s" % domain
   retries = 0
   while true:
   try:
res = srch.parse_results()
break
   except SearchError:
if retries < max_retries:
time.sleep(10)
retries += 1
else:
raise TooManyRetriesError
   print >> k, res.total_results_available
   i = i + 1


-- 
André Engels, andreeng...@gmail.com
--
http://mail.python.org/mailman/listinfo/python-list

Re: While loop

[Chris Rebert]

On Thu, Mar 5, 2009 at 9:49 AM, Fab86  wrote:
> On Mar 5, 5:23 pm, Marco Mariani  wrote:
>> Fab86 wrote:
>> > Is it possible to get the program to catch the exception, wait 10
>> > seconds, then carry of from where it was rather than starting again?

> using sleep and then continue just makes the search start from the
> first search term like before.. Would it be easier to understand if I
> posted sections of my code?
>
> i = 0
> k = open('blah', 'w')
> domains = ["au", "ca", "nl", "be", "...]
> srch = WebSearch(app_id=YahooKey)
>
> while i<200:
>    try:
>        for domain in domains:
>            srch.query = "test site:.%s" % domain
>            res = srch.parse_results()
>            print >> k, res.total_results_available
>            i = i + 1
>
>    except SearchError:
>
> (I currently close then reopen document here then restart i to 0)
>
> Any ideas?

You need to narrow the `try` and put it inside another while-loop:

while i<200:
for domain in domains:
srch.query = "test site:.%s" % domain
while True:
try:
res = srch.parse_results()
except SearchError:
pass
else:
break
print >> k, res.total_results_available
i += 1

Cheers,
Chris

-- 
I have a blog:
http://blog.rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list