Re: C-API: Extract information from function object

[Stefan Behnel]

Gabriel Genellina, 24.03.2010 17:49:

En Wed, 24 Mar 2010 12:09:27 -0300, moerchendiser2k3 escribió:


I have a reference to a function and would like to know how to extract
information from a function object.

Information I am looking for: line and file where this function is
from.

PyObject_Call can do this to when I call a function object and
something failed so these information are written to the traceback. So
any suggestions how to do that?


See the Language Reference; the associated code object holds the file
and starting line the function comes from.
The traceback object contains better information (like the line being
executed instead of the first one). Why don't you use it instead?


Note that this was a follow-up to another recent thread where I managed to 
extract the details from the OP by stoically asking back, especially the 
information that this is not about exception handling but rather about 
reporting incorrect return values.


This is a good example when you need a reason why not to split threads on 
the same subject...


Stefan

--
http://mail.python.org/mailman/listinfo/python-list

Re: sum for sequences?

[Steven D'Aprano]

On Wed, 24 Mar 2010 23:50:23 -0700, TomF wrote:

> On 2010-03-24 14:07:24 -0700, Steven D'Aprano
>  said:
>> On Wed, 24 Mar 2010 15:29:07 +, kj wrote:
>> 
>>> Is there a sequence-oriented equivalent to the sum built-in?  E.g.:
>>> 
>>> seq_sum(((1, 2), (5, 6))) --> (1, 2) + (5, 6) --> (1, 2, 5, 6)
>>> 
>>> ?
>> 
>> Yes, sum.
>> 
>> help(sum) is your friend.
> 
> You might not want to be so glib.  The sum doc sure doesn't sound like
> it should work on lists.
> 
> Returns the sum of a sequence of numbers (NOT strings) plus the
> value of parameter 'start' (which defaults to 0).


What part of that suggested to you that sum might not be polymorphic? 
Sure, it says numbers (which should be changed, in my opinion), but it 
doesn't specify what sort of numbers -- ints, floats, or custom types 
that have an __add__ method. It also singles out strings as excluded. Why 
would you need to explicitly exclude strings, since they're not numbers, 
if sum *only* works with numbers?

E.g. help(math.sin) could have said this, but doesn't:

Return the sine of x (NOT a dictionary)

It doesn't need to, because dicts aren't exceptional: sin doesn't work on 
anything *but* numbers. There's no __sin__ method to call on arbitrary 
types.

The fact that sum does single out strings is a clear sign that strings 
are treated as exceptional and suggests strongly that summing arbitrary 
types should work. I'm not saying that help(sum) explicitly states that 
it works with lists (it clearly doesn't), but it does suggest the 
possibility and makes the experiment worth trying.

I'll also note that the Fine Manual makes it even more clear that sum is 
polymorphic:

http://docs.python.org/library/functions.html#sum




-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Saving a page loaded using the webbrowser library?

[Dr. Benjamin David Clarke]

Does anyone know of a way to save the a loaded web page to file after
opening it with a webbrowser.open() call?

Specifically, what I want to do is get the raw HTML from a web page.
This web page uses Javascript. I need the resulting HTML after the
Javascript has been run. I've seen a lot about trying to get Python to
run Javascript but there doesn't seem to be any promising solution. I
can get the raw HTML that I want by saving the page after it has been
loaded via the webbrowser.open() call. Is there any way to automate
this? Does anyone have any ideas for better approaches to this
problem? I don't need ti to be pretty or anything.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is there any library for indexing binary data?

[Irmen de Jong]

On 3/25/10 4:28 AM, 甜瓜 wrote:

Howdy,

Recently, I am finding a good library for build index on binary data.
Xapian&  Lucene for python binding focus on text digestion rather than
binary data. Could anyone give me some recommendation? Is there any
library for indexing binary data no matter whether it is written in
python?

In my case, there is a very big datatable which stores structured
binary data, eg:
struct Item
{
 long id; // used as key
 double value;
};

I want to build the index on "id" field to speed on searching. Since
this datatable is not constant, the library should support incremental
indexing. If there is no suitable library, I have to do the index by
myself...

Thank you in advance.

--
ShenLei


Put it into an Sqlite database? Or something else from 
http://docs.python.org/library/persistence.html.
Or maybe http://www.pytables.org/ is more suitable to your needs (never 
used that one myself though).
Or install a bank or 2 of memory in your box and read everything into 
memory in one big hashtable.


Btw if you already have a big datatable in which the data is stored, I'm 
guessing that already is in some form of database format. Can't you 
write something that understands that database format.


But I think you need to provide some more details about your data set.

-irmen
--
http://mail.python.org/mailman/listinfo/python-list

Re: Saving a page loaded using the webbrowser library?

[Irmen de Jong]

On 3/25/10 8:41 AM, Dr. Benjamin David Clarke wrote:

Does anyone know of a way to save the a loaded web page to file after
opening it with a webbrowser.open() call?

Specifically, what I want to do is get the raw HTML from a web page.
This web page uses Javascript. I need the resulting HTML after the
Javascript has been run. I've seen a lot about trying to get Python to
run Javascript but there doesn't seem to be any promising solution. I
can get the raw HTML that I want by saving the page after it has been
loaded via the webbrowser.open() call. Is there any way to automate
this? Does anyone have any ideas for better approaches to this
problem? I don't need ti to be pretty or anything.


I think I would use an appropriate GUI automation library to simulate 
user interaction with the web browser that you just started, and e.g. 
select the File > Save page as > HTML only  menu option from the browser...


If the javascript heavily modifies the DOM, that might not work however. 
You might need additional tooling such as Web Developer Toolbar for 
Firefox where you then can View Source > View Generated Source.


irmen

--
http://mail.python.org/mailman/listinfo/python-list

Re: Is there any library for indexing binary data?

[Paul Rubin]

甜瓜  writes:
> Well, Database is not proper because 1. the table is very big (~10^9
> rows) 2. we should support very fast *simple* query that is to get
> value corresponding to single key (~10^7 queries / second).

Just one numeric key/value pair in each row?  What's wrong with
universal hashing?

PyJudy might also be of interest:
  http://www.dalkescientific.com/Python/PyJudy.html
-- 
http://mail.python.org/mailman/listinfo/python-list

how to display variables down a stackdump when an exception occurred

[News123]

Hi,

I captured a piece of code with a try except statement:

In the except part I display a stackdump


try:
domyxmlrpcstuff()
except Exception as e:
import traceback
ex_type,ex_value,e_b = sys.exc_info()
tbstring = traceback.format_exc()
print '%s%s:%s:%s' % \
(msg,ex_type,ex_value,tbstring)


The output, that I receive is:
  File "C:\mycode\myrpcclient.py", line 63, in upload_chunk
rslt = myrpcclient.call()
  File "C:\Python26\lib\xmlrpclib.py", line 1199, in __call__
return self.__send(self.__name, args)
  File "C:\Python26\lib\xmlrpclib.py", line 1489, in __request
verbose=self.__verbose
  File "C:\Python26\lib\xmlrpclib.py", line 1253, in request
return self._parse_response(h.getfile(), sock)
  File "C:\Python26\lib\xmlrpclib.py", line 1387, in _parse_response
p.feed(response)
  File "C:\Python26\lib\xmlrpclib.py", line 601, in feed
self._parser.Parse(data, 0)
ExpatError: syntax error: line 1, column 0

In order to understand more I would like to display the value of
data in C:\Python26\lib\xmlrpclib.py", line 601

Is this possible in a non interactive fashion?

This is a generic question about inspecting variables down the stack,
whenever an exception occurs.

I started another thread specifically about displaying the invalid
xmlrpc data.




N
-- 
http://mail.python.org/mailman/listinfo/python-list

xpat error in xmlrp client. How to inspect data.

[News123]

Hi,

I'm havign a small xmlrpc client, which works normally fine.
(xmlrpc via https)

Sometimes however I receive an Exception about an expat error.


The output, that I receive is:
  File "C:\mycode\myrpcclient.py", line 63, in upload_chunk
rslt = myrpcclient.call()
  File "C:\Python26\lib\xmlrpclib.py", line 1199, in __call__
return self.__send(self.__name, args)
  File "C:\Python26\lib\xmlrpclib.py", line 1489, in __request
verbose=self.__verbose
  File "C:\Python26\lib\xmlrpclib.py", line 1253, in request
return self._parse_response(h.getfile(), sock)
  File "C:\Python26\lib\xmlrpclib.py", line 1387, in _parse_response
p.feed(response)
  File "C:\Python26\lib\xmlrpclib.py", line 601, in feed
self._parser.Parse(data, 0)
ExpatError: syntax error: line 1, column 0


In order to continue debugging I'd like to dump the received http data,
which "C:\Python26\lib\xmlrpclib.py", line 601 tried to parse without
succes.

How can I do this?

thanks for any suggestions


N


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is there any library for indexing binary data?

[甜瓜]

Thank you Rubin! Let me have a look at Judy. It seems good at first glance.

--
ShenLei

2010/3/25 Paul Rubin :
> 甜瓜  writes:
>> Well, Database is not proper because 1. the table is very big (~10^9
>> rows) 2. we should support very fast *simple* query that is to get
>> value corresponding to single key (~10^7 queries / second).
>
> Just one numeric key/value pair in each row?  What's wrong with
> universal hashing?
>
> PyJudy might also be of interest:
>  http://www.dalkescientific.com/Python/PyJudy.html
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is it possible to use re2 from Python?

[Tim Wintle]

On Wed, 2010-03-24 at 10:44 -0700, _wolf wrote:
> yes we can! http://github.com/facebook/pyre2

I had made a thin wrapper experiment with here - looks like the version
he's shipped is relatively complete and compatible with the re module
though.

I'll be interested in seeing how well it performs - The wrapper I had
been experimenting with ended up far slower than the re module for
simple expressions - and the fastest codepaths in RE2 seem to be ones
that are incompatible with the API we're used to using in the re module.

Tim

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is there any library for indexing binary data?

[甜瓜]

Thank you irmen. I will take a look at pytable.
FYI, let me explain the case clearly.

Originally, my big data table is simply array of Item:
struct Item
{
long id;// used as key
BYTE payload[LEN];   // corresponding value with fixed length
};

All items are stored in one file by using "stdio.h" function:
fwrite(itemarray, sizeof(Item), num_of_items, fp);

Note that "id" is randomly unique without any order. To speed up
searching  I regrouped / sorted them into two-level hash tables (in
the form of files).  I want to employ certain library to help me index
this table.

Since the table contains about 10^9 items and LEN is about 2KB, it is
impossible to hold all data in memory. Furthermore, some new item may
be inserted into the array. Therefore incremental indexing feature is
needed.

Hope this help you to understand my case.

--
ShenLei


2010/3/25 Irmen de Jong :
> On 3/25/10 4:28 AM, 甜瓜 wrote:
>>
>> Howdy,
>>
>> Recently, I am finding a good library for build index on binary data.
>> Xapian&  Lucene for python binding focus on text digestion rather than
>> binary data. Could anyone give me some recommendation? Is there any
>> library for indexing binary data no matter whether it is written in
>> python?
>>
>> In my case, there is a very big datatable which stores structured
>> binary data, eg:
>> struct Item
>> {
>> long id; // used as key
>> double value;
>> };
>>
>> I want to build the index on "id" field to speed on searching. Since
>> this datatable is not constant, the library should support incremental
>> indexing. If there is no suitable library, I have to do the index by
>> myself...
>>
>> Thank you in advance.
>>
>> --
>> ShenLei
>
> Put it into an Sqlite database? Or something else from
> http://docs.python.org/library/persistence.html.
> Or maybe http://www.pytables.org/ is more suitable to your needs (never used
> that one myself though).
> Or install a bank or 2 of memory in your box and read everything into memory
> in one big hashtable.
>
> Btw if you already have a big datatable in which the data is stored, I'm
> guessing that already is in some form of database format. Can't you write
> something that understands that database format.
>
> But I think you need to provide some more details about your data set.
>
> -irmen
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: threads (specifically timers) and releasing resources

[Tim Golden]

On 25/03/2010 02:31, Alex Hall wrote:

Okay, I have my program and it has three different modes (there will
be more than that). Each mode will have a timer attached to it. If the
mode remains active and the timer runs out, a function specific to
that mode is called. If that mode is switched away from, however, the
timer is canceled and a new timer is created for the mode to which the
user just switched.


I assume you're using Python's threading.Timer objects as you'd discussed
those before. If so, that's basically a threading.Thread in disguise.
In which case, you're going to have to make sure it cleans up after itself,
releasing whatever resources it holds.

Python's reference-count semantics and cyclic gc will take care of
things in the normal way once the timer-thread has completed. But
you'll have to make sure it completes.


If the latter, is there a way to completely destroy a thread?


No: in Python, a thread has to self-destruct. This is a relatively
FAQ and there are quite a few recipes around. Here's an example of
something which seems to be close to your current needs:

http://code.activestate.com/recipes/464959-resettable-timer-class/

TJG
--
http://mail.python.org/mailman/listinfo/python-list

from import and __init__.py

[egbert]

When I do 'from some_package import some_module'
the __init__.py of some_package will be run.
However, there will not be anything like a  package-module, 
and the effects of __init__.py seem all to be lost. Is that true ?
Or can I still do something useful with __init__.py ?  
e
-- 
Egbert Bouwman - Keizersgracht 197 II - 1016 DS  Amsterdam - 020 6257991

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: the Python Foundation

[Rami Chowdhury]

On 2010-03-24 16:22, Paul Rubin wrote:
> "Steve Holden, Chairman, PSF"  writes:
> > We have also registered the trademark "Python" for use in reference to
> > computer programming languages, thereby ensuring that we can take action
> > should some ill-advised individual or organization decide to produce
> > another language with "Python" in its name
> 
> There has been a Lisp compiler called "Python" for many years:
> 
>  http://portal.acm.org/citation.cfm?id=141471.141558

The FAQ disclaims any connection and claims that Python the Lisp
compiler has been around longer than Python the language...

http://www.cons.org/cmucl/FAQ.html

I was awfully confused the first time I came across SBCL giving me warnings
about Python, though -- proves how well the PSF has safeguarded the name!

> -- 
> http://mail.python.org/mailman/listinfo/python-list
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: from import and __init__.py

[Steve Holden]

egbert wrote:
> When I do 'from some_package import some_module'
> the __init__.py of some_package will be run.
> However, there will not be anything like a  package-module, 
> and the effects of __init__.py seem all to be lost. Is that true ?
> Or can I still do something useful with __init__.py ?  
> e

If I understand correctly what you mean byt a "package-module" then
__init__.py is exactly what you are looking for.

Many packages are built with an empty __init__.py because they are
intended mostly to build a tree-like set of namespaces, but __init__.py
*is* run when the package is imported, and its namespace is bound to the
name of the package within the importing program. So if you have code
you want to run when the package is imported, put it there.

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
See PyCon Talks from Atlanta 2010  http://pycon.blip.tv/
Holden Web LLC http://www.holdenweb.com/
UPCOMING EVENTS:http://holdenweb.eventbrite.com/

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: chroot fails with mount point passed to subprocess.Popen?

[newton10471]

Hi Alf,

After doing some more research, including the character-by-character
comparison you suggested (thank you!), I was able to get things
working the way I wanted using the following:

def getInstalledKernelVersion(mountPoint):
linuxFsRoot = mountPoint + "/root"
installedKernelVersionResult = subprocess.Popen(['/usr/sbin/
chroot',linuxFsRoot,'/bin/rpm','-q','kernel-xen'],
stdout=subprocess.PIPE).communicate()[0]
return installedKernelVersionResult


Thanks very much for you help,

-Matt


On Mar 22, 9:33 am, "Alf P. Steinbach"  wrote:
> * newton10471:
>
>
>
> > Hi,
>
> > I'm trying to use subprocess.Popen() to do a Linux chroot to a mount
> > point passed in as a parameter to the following function:
>
> > def getInstalledKernelVersion(mountPoint):
> >     linuxFsRoot = mountPoint + "/root"
> >     print "type of linuxFsRoot is %s" % type(linuxFsRoot)
> >     installedKernelVersionResult =
> > subprocess.Popen(['chroot',linuxFsRoot,'rpm','-q','kernel-xen'])
> >     return installedKernelVersionResult
>
> > and it dies with the following:
>
> > type of linuxFsRoot is 
> > chroot: cannot change root directory to /storage/mounts/
> > mnt_3786314034939740895.mnt/root: No such file or directory
>
> > When I explicitly set linuxFsRoot = "/storage/mounts/
> > mnt_3786314034939740895.mnt/root", it works fine.
>
> > I also tried this to concatenate the mountpoint + /root, and it failed
> > in the same way:
>
> > linuxFsRoot = ("%s/root") % mountPoint
>
> Use the os.path functions.
>
> > Anyone know what might be happening here?
>
> Since the computed and literal paths /look/ identical and same type, the only
> thing I can imagine is that there is some invisible character. Try comparing 
> the
> computed and literal path character by character. Print the difference or if
> they're identical, that they are.
>
> Possibly you have GIGO problem.
>
> Cheers & hth.,
>
> - Alf

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: chroot fails with mount point passed to subprocess.Popen?

[newton10471]

Hi Alf,

After doing some more research, including the character-by-character
comparison you suggested (thank you!), I was able to get things
working the way I wanted using the following:

def getInstalledKernelVersion(mountPoint):
linuxFsRoot = mountPoint + "/root"
installedKernelVersionResult = subprocess.Popen(['/usr/sbin/
chroot',linuxFsRoot,'/bin/rpm','-q','kernel-xen'],
stdout=subprocess.PIPE).communicate()[0]
return installedKernelVersionResult


Thanks very much for you help,

-Matt


On Mar 22, 9:33 am, "Alf P. Steinbach"  wrote:
> * newton10471:
>
>
>
> > Hi,
>
> > I'm trying to use subprocess.Popen() to do a Linux chroot to a mount
> > point passed in as a parameter to the following function:
>
> > def getInstalledKernelVersion(mountPoint):
> >     linuxFsRoot = mountPoint + "/root"
> >     print "type of linuxFsRoot is %s" % type(linuxFsRoot)
> >     installedKernelVersionResult =
> > subprocess.Popen(['chroot',linuxFsRoot,'rpm','-q','kernel-xen'])
> >     return installedKernelVersionResult
>
> > and it dies with the following:
>
> > type of linuxFsRoot is 
> > chroot: cannot change root directory to /storage/mounts/
> > mnt_3786314034939740895.mnt/root: No such file or directory
>
> > When I explicitly set linuxFsRoot = "/storage/mounts/
> > mnt_3786314034939740895.mnt/root", it works fine.
>
> > I also tried this to concatenate the mountpoint + /root, and it failed
> > in the same way:
>
> > linuxFsRoot = ("%s/root") % mountPoint
>
> Use the os.path functions.
>
> > Anyone know what might be happening here?
>
> Since the computed and literal paths /look/ identical and same type, the only
> thing I can imagine is that there is some invisible character. Try comparing 
> the
> computed and literal path character by character. Print the difference or if
> they're identical, that they are.
>
> Possibly you have GIGO problem.
>
> Cheers & hth.,
>
> - Alf

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: cxfreeze on ubuntu 9.10

[Sandy]

I used cxFreeze without any problem on Ubuntu 9.10 32 bit version. I
tried it on a wxPython gui script and
it works fine though I did'n use any setup file.
Just try cxFreeze executable (should be in /usr/bin/) instead of setup
file.

$/usr/bin/cxFreeze my_script.py --target-dir=/what_ever_dir/

what_ever_dir will have the executable file and other libraries needed
to run the executable.

- dksr


On Mar 25, 12:33 am, Waspinator  wrote:
> Hi,
>
> I'm trying to compile a python script on Ubuntu 9.10. It uses the gtk
> toolkit. I tried to run GUI2EXE for a cxfreeze gui, but even after
> installing wxPython in synaptic it still complains about not having
> it.
>
> I also tried to use cxfreeze by itself but the file it produces does
> not run.
>
> I tried to follow the information 
> athttp://cx-freeze.sourceforge.net/cx_Freeze.html
>
> I created a setup.py file that looks like this (from the page)
>
> from cx_Freeze import setup, Executable
>
> setup(
>         name = "gtk_test",
>         version = "0.1",
>         description = "gtk_test",
>         executables = [Executable("gtk_test.py")])
>
> and ran:
>
> python setup.py build
>
> When I try to run the executable I get the following error:
>
> ../build/exe.linux-i686-2.6/library.zip/gtk/_gtk.py:12:
> RuntimeWarning: tp_compare didn't return -1 or -2 for exception
> ImportError: could not import gio
> ImportError: could not import gio
> Traceback (most recent call last):
>   File "/usr/lib/pymodules/python2.6/cx_Freeze/initscripts/
> Console.py", line 29, in 
>     exec code in m.__dict__
>   File "gtk_test.py", line 274, in 
>   File "gtk_test.py", line 228, in main
> AttributeError: 'module' object has no attribute 'Window'
>
> I was thinking of using the 'copy-dependent-files' option but I'm not
> sure how.
>
> Any ideas?
>
> Thanks

-- 
http://mail.python.org/mailman/listinfo/python-list

Advanced Python Programming Oxford Lectures [was: Re: *Advanced* Python book?]

[Ethan Furman]

Michele Simionato wrote:

On Jan 16, 9:27 pm, mk  wrote:


Hello everyone,

I looked for it I swear, but just can't find it.

Most Python books seem to focus on examples of how to call functions
from standard library. I don't need that, I have online Python
documentation for that.

I mean really advanced mental gymnastics, like gory details of how
Python objects operate, how to exploit its dynamic capabilities, dos and
donts with particular Python objects, advanced tricks, everything from
chained decorators to metaprogramming. Dive Into Python comes closest to
this ideal from what I have found, but still not far enough.

Anybody found such holy grail?


"Expert Python Programming" by Tarek Ziadé is quite good and I wrote
a review for it:

http://www.artima.com/weblogs/viewpost.jsp?thread=240415

There is plenty of info about Advanced Python on the net, much more
than in book form. Come to think of it, there are my Oxford lectures
(the title was exactly "Advanced Python Programming") and I could
republish it on my blog, since I cannot find them on the net anymore.

   Michele Simionato


Michele,

Was wondering if you'd had a chance to re-post your lectures -- just did 
a search for them and came up empty, and I would love to read them!


Many thanks in advance!

~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: sum for sequences?

[Neil Cerutti]

On 2010-03-25, Steven D'Aprano  wrote:
>> You might not want to be so glib.  The sum doc sure doesn't
>> sound like it should work on lists.
>> 
>> Returns the sum of a sequence of numbers (NOT strings) plus the
>> value of parameter 'start' (which defaults to 0).
>
> What part of that suggested to you that sum might not be polymorphic? 
> Sure, it says numbers (which should be changed, in my opinion), but it 
> doesn't specify what sort of numbers -- ints, floats, or custom types 
> that have an __add__ method.

WTF.

-- 
Neil Cerutti
"It's not fun to build walls. But it's even less fun to live
without walls in a world full of zombies." --Greedy Goblin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sum for sequences?

[Stefan Behnel]

Neil Cerutti, 25.03.2010 13:37:

On 2010-03-25, Steven D'Aprano wrote:

You might not want to be so glib.  The sum doc sure doesn't
sound like it should work on lists.

 Returns the sum of a sequence of numbers (NOT strings) plus the
 value of parameter 'start' (which defaults to 0).


What part of that suggested to you that sum might not be polymorphic?
Sure, it says numbers (which should be changed, in my opinion), but it
doesn't specify what sort of numbers -- ints, floats, or custom types
that have an __add__ method.


WTF.


Warning: truth found!

Stefan

--
http://mail.python.org/mailman/listinfo/python-list

Does a tiny Python distribution exist?

[Johny]

Does anyone know if there is a tiny Python distribution available
running in a Linux environment?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sum for sequences?

[Alf P. Steinbach]

* Neil Cerutti:

On 2010-03-25, Steven D'Aprano  wrote:

You might not want to be so glib.  The sum doc sure doesn't
sound like it should work on lists.

Returns the sum of a sequence of numbers (NOT strings) plus the
value of parameter 'start' (which defaults to 0).
What part of that suggested to you that sum might not be polymorphic? 
Sure, it says numbers (which should be changed, in my opinion), but it 
doesn't specify what sort of numbers -- ints, floats, or custom types 
that have an __add__ method.


WTF.


I think Steven's argument is that it would be pointless for 'sum' to distinguish 
between user-defined numerical types and other types that happen to support '+'.


It could make such a distinction since there's a type hierarchy for numbers, but 
then that should IMHO be more clearly documented.


However, given that it isn't restricted to numbers, the restriction wrt. strings 
is a bit perplexing in the context of modern CPython. But for Python 
implementations that don't offer the '+=' optimization it might help to avoid 
gross inefficiencies, namely quadratic time string concatenation. E.g., here's a 
natural implementation of sum  --  with unoptimized '+=' yielding quadratic time 
for the string concatenation (with modern CPython it's linear time, though):



  >>> def sum_all( values, start = 0 ):
  ... s = start
  ... for v in values: s += v
  ... return s
  ...
  >>> sum_all( (1, 2, 3, 4) )
  10
  >>> sum_all( ("a", "b", "c", "d"), "" )
  'abcd'
  >>> sum_all( ((1, 2), (3, 4), (5, 6)), () )
  (1, 2, 3, 4, 5, 6)
  >>> _


However, if that hypothesis about the rationale is correct, then 'sum' should 
also be restricted to not handle tuples or lists, so forth, but at least the 
CPython implementation does.


So perhaps the documentation needs to be more clear?


Cheers,

- Alf
--
http://mail.python.org/mailman/listinfo/python-list

Re: Advice needed on parallel processing in python

[bobicanprogram]

On Mar 24, 1:13 pm, Jon Clements  wrote:
> On 24 Mar, 15:27, Glazner  wrote:
>
>
>
> > Hi!
>
> > I need to replace an app that does number crunching over a local
> > network.
> > it have about 50 computers as slaves
> > each computer needs to run COM that will do the "job"
> > right now the system uses MFC threads and DCOM to distribute the load.
>
> > as i said, Now i'm trying to replace this system with python.
> > I already use win32all and read about Pareller Python and Pyro.
>
> > a requirement is that messages can be sent to the remote worker.
>
> > If anyone can share experience I'll love to hear
>
> > Many Thanks,
> > Yoav Glazner
>
> Would Celery suit?http://celeryproject.org/
>
> hth
>
> Jon.


If you can add just one Linux server onto that network you could use
SIMPL-Python to do what you want.

http://www.icanprogram.com/06py/lesson1/lesson1.html

ie. SIMPL messages going seamlessly from Python (Windows) to Python
(Windows) with the SIMPL sandbox being hosted on the Linux node.

Of course if you wanted to add Linux nodes as processing nodes.
Python (Windows) modules could seamlessly exchange messages with those
as well.  Same would be true if you wanted to distribute part of your
app into the cloud.  All your Python code would look virtually
identical in each of these cases.

As an example of an inexpensive Linux node the SIMPL toolkit has been
ported to the ~$100 Sheeva Plug computer (http://www.icanprogram.com/
simpl/plugsimplbin.self.html)  (http://www.plugcomputer.org).

bob
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python is cool!!

[bobicanprogram]

On Mar 23, 11:55 am, Jose Manuel  wrote:
> I have been learning Python, and it is amazing  I am using the
> tutorial that comes with the official distribution.
>
> At the end my goal is to develop applied mathematic in engineering
> applications to be published on the Web, specially on app. oriented to
> simulations and control systems, I was about to start learning Java
> but I found Python which seems easier to learn that Java.
>
> Would it be easy to integrate Python in Web pages with HTML? I have
> read many info on Internet saying it is, and I hope so 
>
> Any opinion


You probably want to take a look at this tutorial as well:

http://www.icanprogram.com/06py/lesson1/lesson1.html

The SIMPL toolkit will give you added flexibility to choose the
language for some of your more computationally intensive simulations
and still present a unified Python interface to the whole thing.

bob
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Advanced Python Programming Oxford Lectures [was: Re: *Advanced* Python book?]

[Michele Simionato]

On Mar 25, 1:28 pm, Ethan Furman  wrote:
>
> Michele,
>
> Was wondering if you'd had a chance to re-post your lectures -- just did
> a search for them and came up empty, and I would love to read them!
>
> Many thanks in advance!

Oops, I forgot! I will try to make them available soon.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Advice Criticism on Python App

[Bruno Desthuilliers]

Steven D'Aprano a écrit :

On Wed, 24 Mar 2010 21:14:23 -0700, Tim Roberts wrote:


Jimbo  wrote:

class stock:
   code = ""
   purchasePrice= 0
   purchaseQuantity = 0
   price= []  # list of recent prices 
   recentBid= []  # list of recent bids for stock

   recentOffer  = []  # list of recent offers for stock
   stockVol = []  # list of stock quantity available on market

Using lists as class variables is a very good way to create very
surprising bugs.  Consider the following:

[snip]


Don't you think that the class attributes are *supposed* to be shared by 
all instances? In that case the behaviour you show is not a bug at all.


Python's class attributes are indeed supposed to be shared - that's even 
the whole point of having class attributes.


But this feature has proven to be confusing for newcomers that more 
often than not have previous exposure to OOPLs where you do "declare" 
your class "schema" at the class level (where Python defines class 
attributes).


Now reread the OP's code, and you'll find out he's indeed yet another 
victim of this gotcha:


"""
for row in cur.fetchall():
newStock = stock()
newStock.code = row[0]
newStock.purchasePrice= row[1]
newStock.purchaseQuantity = row[2]
cur.execute(stockQuery,[newStock.code])
for rw in cur.fetchall():
newStock.price.append(rw[0])
newStock.recentOffer.append(rw[1])
newStock.recentBid.append(rw[2])
newStock.stockVol.append(rw[3])
   stockList.append(newStock)
"""
--
http://mail.python.org/mailman/listinfo/python-list

Re: threads (specifically timers) and releasing resources

[Alex Hall]

Thanks, this should work.

On 3/25/10, Tim Golden  wrote:
> On 25/03/2010 02:31, Alex Hall wrote:
>> Okay, I have my program and it has three different modes (there will
>> be more than that). Each mode will have a timer attached to it. If the
>> mode remains active and the timer runs out, a function specific to
>> that mode is called. If that mode is switched away from, however, the
>> timer is canceled and a new timer is created for the mode to which the
>> user just switched.
>
> I assume you're using Python's threading.Timer objects as you'd discussed
> those before. If so, that's basically a threading.Thread in disguise.
> In which case, you're going to have to make sure it cleans up after itself,
> releasing whatever resources it holds.
>
> Python's reference-count semantics and cyclic gc will take care of
> things in the normal way once the timer-thread has completed. But
> you'll have to make sure it completes.
>
>> If the latter, is there a way to completely destroy a thread?
>
> No: in Python, a thread has to self-destruct. This is a relatively
> FAQ and there are quite a few recipes around. Here's an example of
> something which seems to be close to your current needs:
>
> http://code.activestate.com/recipes/464959-resettable-timer-class/
>
> TJG
> --
> http://mail.python.org/mailman/listinfo/python-list
>


-- 
Have a great day,
Alex (msg sent from GMail website)
mehg...@gmail.com; http://www.facebook.com/mehgcap
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python is cool!!

[Bruno Desthuilliers]

Jose Manuel a écrit :

I have been learning Python, and it is amazing  I am using the
tutorial that comes with the official distribution.

At the end my goal is to develop applied mathematic in engineering
applications to be published on the Web, specially on app. oriented to
simulations and control systems, I was about to start learning Java
but I found Python which seems easier to learn that Java.


Python is indeed quite lightweight when compared to Java. But it has 
it's share of non-obvious features, dark corners, gotchas, and plain 
warts too.



Would it be easy to integrate Python in Web pages with HTML? I have
read many info on Internet saying it is, and I hope so 


If you think of some server-page PHP-like solution, you won't find much 
usable stuff. There are quite a few Python web development toolkits / 
frameworks, but Django is becoming the de facto standard. Arguably not 
the "best" framework (depending on your definition of "best"), but 
certainly one of the most pythonic and well documented around.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Does a tiny Python distribution exist?

[cassiope]

On Mar 25, 5:45 am, Johny  wrote:
> Does anyone know if there is a tiny Python distribution available
> running in a Linux environment?

Debian has a package: "python-minimal".

HTH...
-- 
http://mail.python.org/mailman/listinfo/python-list

nested threading

[Omer Ihsan]

is there anything as "nested threading"that is, call a thread from
within a thread.
in this case how will thread locking take place.

for example initially there were two functions that were called using
threading.Thread. these wont get unlocked unless both of them are done
with whatever they need to do. if say function 2 calls another thread.
then what??

inquisitive:-|
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Does a tiny Python distribution exist?

[Tim Golden]

On 25/03/2010 15:10, cassiope wrote:

On Mar 25, 5:45 am, Johny  wrote:

Does anyone know if there is a tiny Python distribution available
running in a Linux environment?


Debian has a package: "python-minimal".

HTH...


tinypy?

http://www.tinypy.org/

TJG
--
http://mail.python.org/mailman/listinfo/python-list

Represent object type as

[Jason]

Hi,

I want to send objects (new style) over DBUS. DBUS can only send
fairly primitive types[1] so I turn my objects into dicts and send
that. I'm reusing the __getstate__ function I wrote for pickling like
so:

def __getstate__(self):
attrs = self.__dict__.copy()
return attrs

...which is perfectly adequate to dict-ify and reconstitute this
simple class. That works just fine over DBUS.

The trouble is, the object could actually be of some slightly
different types, so I'd like to include that information as well. I
tried something like:

def __getstate__(self):
attrs = self.__dict__.copy()
attrs.update({'type': type(self)})
return attrs

...but then realised that "type" is not primitive enough for DBUS to
pickle.

So, (a) can I get the type name from the type object, or (b) is there
a better way to do this?

(This pertains to Python 2.5.4.)

Cheers,
— Jason

[1] http://dbus.freedesktop.org/doc/dbus-python/doc/tutorial.html#data-types
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Represent object type as

[Bruno Desthuilliers]

Jason a écrit :

Hi,

I want to send objects (new style) over DBUS. DBUS can only send
fairly primitive types[1] so I turn my objects into dicts and send
that. I'm reusing the __getstate__ function I wrote for pickling like
so:

def __getstate__(self):
attrs = self.__dict__.copy()
return attrs

...which is perfectly adequate to dict-ify and reconstitute this
simple class. That works just fine over DBUS.

The trouble is, the object could actually be of some slightly
different types, so I'd like to include that information as well. I
tried something like:

def __getstate__(self):
attrs = self.__dict__.copy()
attrs.update({'type': type(self)})


  attrs['type'] = type(self)

Do the same thing with less work !-)

Also and while we're at it, using a __magicname__ convention here (ie : 
'__type__' instead of 'type') might avoid possible collisions.



return attrs

...but then realised that "type" is not primitive enough for DBUS to
pickle.

So, (a) can I get the type name from the type object,


attrs['__typename__'] = type(self).__name__


Warning: won't be very useful if your code still uses old-style classes.


or (b) is there
a better way to do this?


Depends on what you do with this dict, DBUS etc. And of your definition 
of "better", of course.



HTH
--
http://mail.python.org/mailman/listinfo/python-list

ANN: Pymazon 0.1.1 Released

[Chris Colbert]

I'm happy to announce the release of Pymazon 0.1.1!

This release brings a big enhancement in the form of PyGtk support in
addition to the PyQt4 and Command line interfaces already available.
A special thanks to Ray Meyers for his gtk commits!

Pymazon Changelog

0.1.1
-
- Added support for command line options and configuration file
- Added support for save name templates
- Added a threaded downloader which runs a user-specified simultaneous
threads when in gui mode
- Added a PyGtk Gui (Submitted by Raymond Myers)
- Rewrote the QT Gui with Qt Designer and pyuic4 - this simplified and
cleaned up a bunch of stuff
- Added graphical progress bars to the Gui's
- Removed a check that asserted the downloaded file size was the same as
specified in the amz file
  as Amazon was misreporting file size and it was causing Pymazon to
erroneously fail
- Cleaned up code all over the place: logging, settings, etc...


Pymazon is available in the cheeseshop and google code.

Pymazon is a Python implemented downloader for the amazon mp3 store.

You can read more about Pymazon at http://code.google.com/p/pymazon/.

I always appreciate comments and bug reports.

Cheers!

Chris
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: from import and __init__.py

[Terry Reedy]

On 3/25/2010 6:16 AM, egbert wrote:

When I do 'from some_package import some_module'
the __init__.py of some_package will be run.
However, there will not be anything like a  package-module,
and the effects of __init__.py seem all to be lost. Is that true ?


No. If you do

from sys import modules
print(modules.keys())

you will see both some_package and some_package.some_module among the 
entries. The first is the result of executing some_package/__init__.py. 
As usual, that code will *not* be re-exectured on subsequent imports 
involving some_package.



> Or can I still do something useful with __init__.py ?
> e

Some packages put something like 'from _default_stuff import *' in 
__init__.py with the intention that the package by used as


import package

perhaps [optionally] followed by

import package.specialized_stuff

Terry Jan Reedy



--
http://mail.python.org/mailman/listinfo/python-list

ANN: ActivePython 2.6.5.12 and 3.1.2.3 are now available

[Sridhar Ratnakumar]

We are pleased to announce the availability of both ActivePython 2.6.5.12 and 
ActivePython 3.1.2.3. 

http://www.activestate.com/activepython/

Here is what you should know about these two releases:

PyWin32: PyWin32 is now included in the 64-bit & Python3 builds! Since we 
recently updated our PyWin32 source tree, this release comes with several bug 
fixes.

PyPM: There is now an “upgrade” feature in PyPM, our beloved Python Package 
Manager distributed with ActivePython free of charge. What this means is that 
if you type “pypm upgrade” it will update all your installed Python Packages to 
the latest version to save you time and keep you up-to-date.

Switch between Python versions: Also new in this 2.6 release is a new tool 
called "pythonselect" that can be used to switch between multiple ActivePython 
versions. This is currently only supported on MacOSX although support for other 
platforms and, perhaps, other Python installations, is in the roadmap (patches 
are welcome too!).

Dive Into Python 3: ActivePython 3.1.2.3 now includes Mark Pilgrim’s Dive Into 
Python 3, the popular advanced tutorial for learning Python3. You can find it  
in the documentation section.

Miscellaneous: We also updated the base Tcl/Tk version to 8.5.8; added OpenSSL 
and ctypes support in the 64-bit build; upgraded to openssl-0.9.8l; included 
Tcl/Tk development headers – among several other bug fixes with IDLE and 
Tkinter installation. And of course, they use the latest Python version: 2.6.5 
and 3.1.2.

See the release notes for full details:

http://docs.activestate.com/activepython/2.6/relnotes.html#changes
http://docs.activestate.com/activepython/2.6/relnotes.html#changes

And our blog post on this release:


http://blogs.activestate.com/2010/03/activestate-releases-activepython-2-6-5-12-and-activepython-3-1-2-3/


What is ActivePython?
-

ActivePython is ActiveState's binary distribution of Python. Builds for 
Windows, Mac OS X, Linux are made freely available. Builds for Solaris, HP-UX 
and AIX, and access to older versions are available with ActivePython Business 
Edition:

http://www.activestate.com/business_edition/

ActivePython includes the Python core and the many core extensions: zlib and 
bzip2 for data compression, the Berkeley DB (bsddb) and SQLite (sqlite3) 
database libraries, OpenSSL bindings for HTTPS support, the Tix GUI widgets for 
Tkinter, ElementTree for XML
processing, ctypes (on supported platforms) for low-level library access, and 
others. The Windows distribution ships with PyWin32 -- a
suite of Windows tools developed by Mark Hammond, including bindings to the 
Win32 API and Windows COM. 

Beginning with the 2.6.3.7 release, ActivePython includes a binary package 
manager for Python (PyPM) that can be used to install packages much easily. For 
example:

pypm install pylons

See this page for full details:

http://docs.activestate.com/activepython/2.6/whatsincluded.html

As well, ActivePython ships with a wealth of documentation for both new and 
experienced Python programmers. In addition to the core Python docs, 
ActivePython includes the "What's New in Python" series, "Dive into Python", 
the Python FAQs & HOWTOs, and the Python Enhancement Proposals (PEPs).

An online version of the docs can be found here:

http://docs.activestate.com/activepython/2.6/

We would welcome any and all feedback to:

activepython-feedb...@activestate.com

Please file bugs against ActivePython at:

http://bugs.activestate.com/query.cgi?set_product=ActivePython


On what platforms does ActivePython run?


ActivePython includes installers for the following platforms:

- Windows/x86
- Windows/x64 (aka "AMD64")
- Mac OS X
- Linux/x86
- Linux/x86_64 (aka "AMD64")
- Solaris/SPARC (Business Edition only)
- Solaris/x86 (Business Edition only)
- HP-UX/PA-RISC (Business Edition only)
- AIX/PowerPC (Business Edition only)
- AIX/PowerPC 64-bit (Business Edition only)

Custom builds are available in Enterprise Edition:

http://www.activestate.com/activepython/enterprise/


Thanks, and enjoy!

The Python Team

--
Sridhar Ratnakumar
sridharr at activestate.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Repetition of work in Jython

[Andrey Fedorov]

Hi all,

So from what I understand, Jython translates Python code into JVM byte code.
Does anyone know why this was chosen instead of translating Python bytecode
to JVM bytecode directly? It seems that it would be a lot easier to get
Jython up-to-speed if there could be some "shared components" between them
and CPython, no?

- Andrey
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Castrated traceback in sys.exc_info()

[Andreas Löscher]

> As you see, the traceback only starts from function c, which handles the 
> exception.
> It doesn't show main(), a() and b(), which might however be (and are, in 
> my case) critical to diagnose the severity of the problem (since many 
> different paths would lead to calling c()).

This results in the way the tb is created.

If an exception occours, and there is no one who handles it in the
current function, a traceback object is created. Because the the calling
function could have a try-except clause, the function returns with NULL
and an set exception. Now if there is no one to handle the exception, a
traceback object is created and linked with the existing one.

In your case does the traceback start from the function c, because no
exception occoured in main() etc. You catched it. If you want to
determine the calling path, you can take a look at the encapsuled frame
in the tb. 

Best


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Repetition of work in Jython

[Steve Holden]

Andrey Fedorov wrote:
> Hi all,
> 
> So from what I understand, Jython translates Python code into JVM byte
> code. Does anyone know why this was chosen instead of translating Python
> bytecode to JVM bytecode directly? It seems that it would be a lot
> easier to get Jython up-to-speed if there could be some "shared
> components" between them and CPython, no?
> 
I don't *know* why Jim Hugunin chose not to take that approach, but I
can easily imagine it would be because a mapping of the Python byte code
to the Java byte code would be much less efficient.

There are various approaches that one could take. The least efficient
would be to write a Python virtual machine in Java, thereby putting two
levels of interpretation into the execution paths.

I doubt it would be possible to get maximal JVM efficiency from code
cast into Python byte codes just because of the disparity between the
two VMs.

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
See PyCon Talks from Atlanta 2010  http://pycon.blip.tv/
Holden Web LLC http://www.holdenweb.com/
UPCOMING EVENTS:http://holdenweb.eventbrite.com/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Possible? Python 2.6.x and PythonWin on 64-bit Windows 7

[Sridhar Ratnakumar]

On 2010-02-09, at 2:49 PM, Sridhar Ratnakumar wrote:

> 
> On 2010-02-07, at 5:02 PM, escalation746 wrote:
> 
>> Andrej Mitrovic wrote:
>> 
>>> Perhaps you've accidentally downloaded the wrong version of PythonWin?
>> 
>> Erk, yes, my bad.
>> 
>> Thanks for the quick help! Though I still wonder why the ActiveState
>> build does not include this.
> 
> I just added support for PyWin32 in ActivePython 2.6 64-bit.
> 
>  http://twitter.com/sridhr/status/8874549314
> 
> I should be part of next release (2.6.4.11).

PyWin32, and thus PythonWin, is now available in all of the recent ActivePython 
releases: 2.5.5.7, 2.6.5.12 and 3.1.2.3. 

See also 
http://blogs.activestate.com/2010/03/activestate-releases-activepython-2-6-5-12-and-activepython-3-1-2-3/

-srid

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is there any library for indexing binary data?

[Irmen de Jong]

On 25-3-2010 10:55, 甜瓜 wrote:
> Thank you irmen. I will take a look at pytable.
> FYI, let me explain the case clearly.
> 
> Originally, my big data table is simply array of Item:
> struct Item
> {
>  long id;// used as key
>  BYTE payload[LEN];   // corresponding value with fixed length
> };
> 
> All items are stored in one file by using "stdio.h" function:
>  fwrite(itemarray, sizeof(Item), num_of_items, fp);
> 
> Note that "id" is randomly unique without any order. To speed up
> searching  I regrouped / sorted them into two-level hash tables (in
> the form of files).  I want to employ certain library to help me index
> this table.
> 
> Since the table contains about 10^9 items and LEN is about 2KB, it is
> impossible to hold all data in memory. Furthermore, some new item may
> be inserted into the array. Therefore incremental indexing feature is
> needed.

I see, I thought the payload data was small as well. What about this idea:
Build a hash table where the keys are the id from your Item structs and
the value is the file seek offset of the Item 'record' in your original
datafile. (although that might generate values of type long, which take
more memory than int, so maybe we should use file_offset/sizeof(Item).
This way you can just keep your original data file (you only have to
scan it to build the hash table) and you will avoid a lengthy conversion
process.

If this hashtable still doesn't fit in memory use a sparse array
implementation of some sort that is more efficient at storing simple
integers, or just put it into a database solution mentioned in earlier
responses.

Another thing: I think that your requirement of 1e7 lookups per second
is a bit steep for any solution where the dataset is not in core memory
at once though.

Irmen.
-- 
http://mail.python.org/mailman/listinfo/python-list

Improved timedelta attributes/methods

[Christian Ştefănescu]

Hello dear Python-wielding developers!

I generally like date/time handling in Python very much, especially how date
operations result in Timedelta
objects.
But I find it somewhat impractical, that you can only get days, seconds and
microseconds out of a time delta. I think it would be much more comfortable
to have fields for minutes, hours, maybe also years, months. Is there a
specific reasoning why this is not available?

I would gladly do my best to implement such a solution, if it would make
sense to the devs.

Regards,
Chris
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RELEASED Python 2.6.5

[Peter]

On Wed, 24 Mar 2010 23:22:01 +0100, Martin v. Loewis wrote:

>> Is anyone else having trouble with the 2.6.5 Windows x86 installer?
> 
> Not me. Run
> 
> msiexec /i py...msi /l*v py.log
> 
> and inspect py.log for errors (post it to bugs.python.org if you can't
> determine the cause of the problems).


Martin,

Thanks for the tip about the msiexec command line usage.  If I run the 
installer without selecting the Advanced compiling option, it works fine.

The relevent part of the log when it fails using the Advanced compiling 
option is as follows:



MSI (s) (4C:B4) [14:41:27:205]: Doing action: CompilePyc
Action 14:41:27: CompilePyc. 
Action start 14:41:27: CompilePyc.
MSI (s) (4C:B4) [14:45:45:528]: Note: 1: 1722 2: CompilePyc 3: C:\bin
\Python26\python.exe 4: -Wi "C:\bin\Python26\Lib\compileall.py" -f -x 
bad_coding|badsyntax|site-packages|py3_ "C:\bin\Python26\Lib" 
MSI (s) (4C:B4) [14:45:45:528]: Note: 1: 2262 2: Error 3: -2147287038 
Error 1722. There is a problem with this Windows Installer package. A 
program run as part of the setup did not finish as expected. Contact your 
support personnel or package vendor.  Action CompilePyc, location: C:\bin
\Python26\python.exe, command: -Wi "C:\bin\Python26\Lib\compileall.py" -f 
-x bad_coding|badsyntax|site-packages|py3_ "C:\bin\Python26\Lib" 
MSI (s) (4C:B4) [14:47:41:133]: Note: 1: 2262 2: Error 3: -2147287038 
MSI (s) (4C:B4) [14:47:41:133]: Product: Python 2.6.5 -- Error 1722. 
There is a problem with this Windows Installer package. A program run as 
part of the setup did not finish as expected. Contact your support 
personnel or package vendor.  Action CompilePyc, location: C:\bin\Python26
\python.exe, command: -Wi "C:\bin\Python26\Lib\compileall.py" -f -x 
bad_coding|badsyntax|site-packages|py3_ "C:\bin\Python26\Lib" 

Action ended 14:47:41: CompilePyc. Return value 3.
Action ended 14:47:41: INSTALL. Return value 3.



I believe the cause of the installation failure message is with the 
syntax of the following command:

C:\bin\Python26\python.exe -Wi "C:\bin\Python26\Lib\compileall.py" -f -x 
bad_coding|badsyntax|site-packages|py3_ "C:\bin\Python26\Lib"

If you run this command in the Windows XP shell, it yields an error.  If 
you wrap the -x option's args in double quotes, it runs ok (except for a 
syntax error when compiling one of the python source files - I don't 
remember which one):

C:\bin\Python26\python.exe -Wi "C:\bin\Python26\Lib\compileall.py" -f -x 
"bad_coding|badsyntax|site-packages|py3_" "C:\bin\Python26\Lib"

So it appears that the Windows XP shell is interpreting the "|" 
characters within the -x option's args as pipe characters and trys to 
pipe the "multiple commands" together.  The simple work around is to not 
use the Advanced compiling option.

Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sum for sequences?

[Steven D'Aprano]

On Thu, 25 Mar 2010 14:02:05 +0100, Alf P. Steinbach wrote:

> * Neil Cerutti:
>> On 2010-03-25, Steven D'Aprano 
>> wrote:
 You might not want to be so glib.  The sum doc sure doesn't sound
 like it should work on lists.

 Returns the sum of a sequence of numbers (NOT strings) plus the
 value of parameter 'start' (which defaults to 0).
>>> What part of that suggested to you that sum might not be polymorphic?
>>> Sure, it says numbers (which should be changed, in my opinion), but it
>>> doesn't specify what sort of numbers -- ints, floats, or custom types
>>> that have an __add__ method.
>> 
>> WTF.
> 
> I think Steven's argument is that it would be pointless for 'sum' to
> distinguish between user-defined numerical types and other types that
> happen to support '+'.

Before Python2.6, which introduced a numeric tower, Python *couldn't* 
reliably distinguish between numeric types and other types that 
overloaded +. Since Python discourages type-checking in favour of duck-
typing and try...except, this is seen as a good thing.

My argument is that sum isn't hard-coded to only work on the built-ins 
ints or floats, but it supports any object that you can use the + 
operator on. The *sole* exceptions are str and unicode (not even 
UserString), and even there it is very simple to overcome the restriction:

>>> sum(['a', 'b'], '')
Traceback (most recent call last):
  File "", line 1, in 
TypeError: sum() can't sum strings [use ''.join(seq) instead]
>>> class S:
... def __add__(self, other):
... return other
...
>>> sum(['a', 'b'], S())
'ab'


[...]
> However, given that it isn't restricted to numbers, the restriction wrt.
> strings is a bit perplexing in the context of modern CPython. But for
> Python implementations that don't offer the '+=' optimization it might
> help to avoid gross inefficiencies, namely quadratic time string
> concatenation.

I agree -- the Python philosophy is to allow the user to shoot themselves 
in the foot if they wish to. You're responsible for the Big Oh behaviour 
of your code, not the compiler.


[...]
> However, if that hypothesis about the rationale is correct, then 'sum'
> should also be restricted to not handle tuples or lists, so forth, but
> at least the CPython implementation does.

The reasoning is that naive users are far, far more likely to try summing 
a large list of strings than to try summing a large list of lists, and 
therefore in practical terms the consequences of allowing sum on lists is 
slight enough and rare enough to not be worth the check.

I suspect that this is just an after the fact rationalisation, and that 
the real reason is that those responsible for the hand-holding in sum 
merely forgot, or didn't know, that repeated addition of lists and tuples 
is also O(N**2). But I've never cared enough to dig through the archives 
to find out.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Revisiting Generators and Subgenerators

[Winston]

I have been reading PEP 380 because I am writing a video game/
simulation in Jython and I need cooperative multitasking. PEP 380 hits
on my problem, but does not quite solve it for me. I have the
following proposal as an alternative to PEP380. I don't know if this
is the right way for me to introduce my idea, but below is my writeup.
Any thoughts?


Proposal for a new Generator Syntax in Python 3K--
A Baton object for generators to allow subfunction to yield, and to
make
them symetric.

Abstract

Generators can be used to make coroutines. But they require the
programmer
to take special care in how he writes his generator. In
particular,
only the generator function may yield a value. We propose a
modification
to generators in Python 3 where a "Baton" object is given to both
sides of a generator. Both sides use the baton object to pass
execution
to the other side, and also to pass values to the other side.

The advantages of a baton object over the current scheme are: (1)
the generator function can pass the baton to a subfunction,
solving the
needs of PEP 380, (2) after creation both sides of the generator
function
are symetric--they both can call yield(), send(), next(). They do
the
same thing. This means programming with generators is the same as
programming with normal functions. No special contortions are
needed
to pass values back up to a yield command at the top.

Motivation
--
Generators make certain programming tasks easier, such as (a) an
iterator
which is of infinite length, (b) using a "trampoline function"
they can
emulate coroutines and cooperative multitasking, (c) they can be
used
to make both sides of a producer-consumer pattern easy to write--
both
sides can appear to be the caller.

On the down side, generators as they currently are implemented in
Python 3.1
require the programmer to take special care in how he writes his
generator. In particular, only the generator function may yield a
value--subfunctions called by the generator function may not yield
a value.

Here are  two use-cases in which generators are commonly used, but
where the
current limitation causes less readable code:

1) a long generator function which the programmer wants to split
into several
functions. The subfunctions should be able to yield a result.
Currently
the subfunctions have to pass values up to the main generator
and have
it yield the results back. Similarly subfunctions cannot
receive values
that the caller sends with generator.send()

2) generators are great for cooperative multitasking. A common use-
case is
agent simulators where many small "tasklets" need to run and
then pass
execution over to other tasklets. Video games are a common
scenario,
as is SimPy. Without cooperative multitasking, each tasklet
must be
contorted to run in a small piece and then return. Generators
help this,
but a more complicated algorithm which is best decomposed into
several
functions must be contorted because the subfuctions cannot
yield or
recive data from the generator.send().

Here is also a nice description of how coroutines make programs
easier to read and write:
http://www.chiark.greenend.org.uk/~sgtatham/coroutines.html

Proposal

If there is a way to make a sub-function of a generator yield and
receive
data from generator.send(), then the two problems above are
solved.

For example, this declares a generator. The first parameter of the
generator
is the "context" which represents the other side of the execution
frame.

a Baton object represents a passing of the execution from one line
of code to another. A program creates a Baton like so:

generator f( baton ):
# compute something
baton.yield( result )
# compute something
baton.yield( result )

baton = f()
while True:
print( baton.yield() )


A generator function, denoted with they keyword "generator"
instead of "def"
will return a "baton". Generators have the following methods:
__call__( args... ) --
This creates a Baton object which is passed back to the
caller,
i.e. the code that executed the Baton() command. Once the
baton
starts working, the two sides are symetric. So we will
call the
first frame, frame A and the code inside 'function' frame
B.
Frame is is returned a baton object. As soon as frame A
calls baton.yield(), frame B begins, i.e. 'function'
starts
to run. function is passed the baton as its first
argument,
and any additional arguments are also passed in. When
frame B
yields, any value that it yields will be returned to frame
A
as the result of it's yield().
Baton

Revisiting Generators and Subgenerators

[Winston]

Here's my proposal again, but hopefully with better formatting so you
can read it easier.


-Winston
-

Proposal for a new Generator Syntax in Python 3K--
A Baton object for generators to allow subfunction to yield, and to
make
them symetric.

Abstract

Generators can be used to make coroutines. But they require
the programmer to take special care in how he writes his
generator. In particular, only the generator function may
yield a value. We propose a modification to generators in
Python 3 where a "Baton" object is given to both sides of a
generator. Both sides use the baton object to pass execution
to the other side, and also to pass values to the other side.

The advantages of a baton object over the current scheme are:
(1) the generator function can pass the baton to a
subfunction, solving the needs of PEP 380, (2) after creation
both sides of the generator function are symetric--they both
can call yield(), send(), next(). They do the same thing.
This means programming with generators is the same as
programming with normal functions. No special contortions are
needed to pass values back up to a yield command at the top.

Motivation
--
Generators make certain programming tasks easier, such as (a)
an iterator which is of infinite length, (b) using a
"trampoline function" they can emulate coroutines and
cooperative multitasking, (c) they can be used to make both
sides of a producer-consumer pattern easy to write--both
sides can appear to be the caller.

On the down side, generators as they currently are
implemented in Python 3.1 require the programmer to take
special care in how he writes his generator. In particular,
only the generator function may yield a value--subfunctions
called by the generator function may not yield a value.

Here are  two use-cases in which generators are commonly
used, but where the current limitation causes less readable
code:

1) a long generator function which the programmer wants to
split into several functions. The subfunctions should be able
to yield a result. Currently the subfunctions have to pass
values up to the main generator and have it yield the results
back. Similarly subfunctions cannot receive values that the
caller sends with generator.send()

2) generators are great for cooperative multitasking. A
common use-case is agent simulators where many small
"tasklets" need to run and then pass execution over to other
tasklets. Video games are a common scenario, as is SimPy.
Without cooperative multitasking, each tasklet must be
contorted to run in a small piece and then return. Generators
help this, but a more complicated algorithm which is best
decomposed into several functions must be contorted because
the subfuctions cannot yield or recive data from the
generator.send().

Here is also a nice description of how coroutines make
programs easier to read and write:
http://www.chiark.greenend.org.uk/~sgtatham/coroutines.html

Proposal

If there is a way to make a sub-function of a generator yield
and receive data from generator.send(), then the two problems
above are solved.

For example, this declares a generator. The first parameter
of the generator is the "context" which represents the other
side of the execution frame.

a Baton object represents a passing of the execution from one
line of code to another. A program creates a Baton like so:

generator f( baton ):
# compute something
baton.yield( result )
# compute something
baton.yield( result )

baton = f()
while True:
print( baton.yield() )


A generator function, denoted with they keyword "generator"
instead of "def" will return a "baton". Generators have the
following methods:
__call__( args... ) --
This creates a Baton object which is passed back to
the caller, i.e. the code that executed the Baton()
command. Once the baton starts working, the two sides
are symetric. So we will call the first frame, frame
A and the code inside 'function' frame B. Frame is is
returned a baton object. As soon as frame A calls
baton.yield(), frame B begins, i.e. 'function' starts
to run. function is passed the baton as its first
argument, and any additional arguments are also
passed in. When frame B yields, any value that it
yields will be returned to frame A as the result of
it's yield().

Batons have the following methods:
yield( arg=None ) -- This method will save the current
execution state, restore the other execution state,
and start running the other function from where it
l

Create a class at run-time

[Michel]

Hi everyone,

I'm trying to dynamically create a class. What I need is to define a
class, add methods to it and later instantiate this class. Methods
need to be bound to the instance though, and that's my problem. Here
is what I have so far:

method_template = "def test_foo(self):\
#actual test_foo\
pass"
exec method_template

TestClass = types.ClassType("MyTestClass", (unittest.TestCase, ), {})
TestClass.__module__ = "test"

now what to do next?
I looked at types.MethodType but it needs an instance to bind the
method and a function object.
Should I define __new__ to bind the method during instantiation?

Hope this makes sense,

Michel.
-- 
http://mail.python.org/mailman/listinfo/python-list

XPCOM/hulahop: waiting for JavaScript finished?

[John Bokma]

I am playing a bit with hulahop/xpcom using the code found at

http://www.advogato.org/article/1014.html

but have no idea how to check if some JavaScript has finised running.
The _loaded method of Browser is called after the document has finished
loading, but as far as I can tell this doesn't mean that the JavaScript
has finised setting things up.

Is there a way to either wait a few seconds in _loaded (fragile, of
course). time.sleep() (as expected) blocks.

Or better, is there a way to check if JavaScript has stopped running?
(Hmmm... thinking of it, this can also run on events, so maybe there is
no good way to do this?)

Thanks,

-- 
John Bokma   j3b

Hacking & Hiking in Mexico -  http://johnbokma.com/
http://castleamber.com/ - Perl & Python Development
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Create a class at run-time

[Michiel Overtoom]

On 2010-03-25 23:00, Michel wrote:


I'm trying to dynamically create a class. What I need is to define a
class, add methods to it and later instantiate this class. Methods
need to be bound to the instance though, and that's my problem.


Maybe this snippet is of any help?

import functools

class Template(object):
pass

def printmyname(self):
print self.name

t=Template()
t.name="Pete"
t.printmyname=functools.partial(printmyname,t)

u=Template()
u.name="Mary"
u.printmyname=functools.partial(printmyname,u)

t.printmyname()
u.printmyname()

Greetings,


--
"The ability of the OSS process to collect and harness
the collective IQ of thousands of individuals across
the Internet is simply amazing." - Vinod Valloppillil
http://www.catb.org/~esr/halloween/halloween4.html
--
http://mail.python.org/mailman/listinfo/python-list

Re: from import and __init__.py

[egbert]

On Thu, Mar 25, 2010 at 12:43:13PM -0400, Terry Reedy wrote:
> On 3/25/2010 6:16 AM, egbert wrote:
> >When I do 'from some_package import some_module'
> >the __init__.py of some_package will be run.
> >However, there will not be anything like a  package-module,
> >and the effects of __init__.py seem all to be lost. Is that true ?
> 
> No. If you do
> 
> from sys import modules
> print(modules.keys())
> 
> you will see both some_package and some_package.some_module among
> the entries. 
Yes, you are right. And I can reach everything with
modules['some_package']
or variants thereof.
Thanks,
egbert

-- 
Egbert Bouwman - Keizersgracht 197 II - 1016 DS  Amsterdam - 020 6257991

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Create a class at run-time

[Patrick Maupin]

On Mar 25, 5:00 pm, Michel  wrote:
> Hi everyone,
>
> I'm trying to dynamically create a class. What I need is to define a
> class, add methods to it and later instantiate this class. Methods
> need to be bound to the instance though, and that's my problem. Here
> is what I have so far:

Well, you should just fill your empty dict with function definitions,
BEFORE you build the class.  That's easiest.  Also, you can just use
type:

def foo(*whatever):
print foo

bar = type('MyDynamicClass', (object,), dict(foo=foo))

HTH,
Pat
-- 
http://mail.python.org/mailman/listinfo/python-list

Python database of plain text editable by notepad or vi

[James Harris]

I am looking to store named pieces of text in a form that can be
edited by a standard editor such as notepad (under Windows) or vi
(under Unix) and then pulled into Python as needed. The usual record
locking and transactions of databases are not required.

Another way to look at it is to treat the separate files as entries in
a dictionary. The file name would be the key and the lines of the file
the value.

Anyone know of a database (with a Python interface) which will allow
text files to be treated as database fields? If not I can just write
it but I thought it best to ask if there was an existing solution
first.

James
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python database of plain text editable by notepad or vi

[jkn]

Kirbybase is one possibility.

http://pypi.python.org/pypi/KirbyBase/1.9


J^n

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python database of plain text editable by notepad or vi

[Jon Clements]

On 25 Mar, 22:40, James Harris  wrote:
> I am looking to store named pieces of text in a form that can be
> edited by a standard editor such as notepad (under Windows) or vi
> (under Unix) and then pulled into Python as needed. The usual record
> locking and transactions of databases are not required.
>
> Another way to look at it is to treat the separate files as entries in
> a dictionary. The file name would be the key and the lines of the file
> the value.
>
> Anyone know of a database (with a Python interface) which will allow
> text files to be treated as database fields? If not I can just write
> it but I thought it best to ask if there was an existing solution
> first.
>
> James

I could be missing something here, but aren't you basically just
talking about an OS's filesystem?

glob or listdir somewhere, then create a dict using the file contents
would meet your criteria, with very little lines of code -- but I
would be interested to know what the use-case was for this... Is it
read completely at start up time, or if each file contains a large
amount of lines and aren't fixed width (or has no other indexing
support without maintenance), then is a complete sequential-scan
required each time, or do you just tell the user to not update when
running (unless I s'pose something along the lines of a SIGHUP for
config files is applicable).

Sorry, just don't understand why you'd want this.

Jon.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: nested threading

[Chris Colbert]

Spawning a thread from within a thread works just fine. Calling
thread.start() is a non-blocking function and returns immediately.

On Thu, Mar 25, 2010 at 11:23 AM, Omer Ihsan  wrote:

> is there anything as "nested threading"that is, call a thread from
> within a thread.
> in this case how will thread locking take place.
>
> for example initially there were two functions that were called using
> threading.Thread. these wont get unlocked unless both of them are done
> with whatever they need to do. if say function 2 calls another thread.
> then what??
>
> inquisitive:-|
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sniffing encoding type by looking at file BOM header

[Lawrence D'Oliveiro]

In message , 
pyt...@bdurham.com wrote:

> BOM_UTF8 = '\xef\xbb\xbf'

Since when does UTF-8 need a BOM?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Sniffing encoding type by looking at file BOM header

[Irmen de Jong]

On 26-3-2010 0:16, Lawrence D'Oliveiro wrote:

In message,
pyt...@bdurham.com wrote:


BOM_UTF8 = '\xef\xbb\xbf'


Since when does UTF-8 need a BOM?


It doesn't, but it is allowed. Not recommended though.
Unfortunately several tools, such as notepad.exe, have a tendency of 
silently adding it when saving files.


-irmen

--
http://mail.python.org/mailman/listinfo/python-list

Don't understand behavior; instance form a class in another class' instance

[Martin P. Hellwig]

Hi all,

When I run the following snippet (drastically simplified, to just show 
what I mean):

>>
import platform, sys

class One(object):
def __init__(self):
self.one = True

def change(self):
self.one = False

class Two(object):
def __init__(self):
self._instance_one = One()
self.one = self._instance_one.one
self.change = self._instance_one.change

if __name__ == '__main__':
print(sys.version)
print(platform.machine())
print(80*'#')
TEST1 = One()
print(TEST1.one)
TEST1.change()
print(TEST1.one)
TEST1 = None
print(80*'#')
TEST2 = Two()
print(TEST2.one)
TEST2.change()
print(TEST2.one
>>

I get the following result:

<<
[GCC 4.2.1 20070719  [FreeBSD]]
amd64

True
False

True
True

<<

What I don't understand why in the second test, the last boolean is True 
instead of (what I expect) False.
Could somebody enlighten me please as this has bitten me before and I am 
confused by this behavior.


Thanks in advance

--
mph
--
http://mail.python.org/mailman/listinfo/python-list

Re: Don't understand behavior; instance form a class in another class' instance

[Christian Heimes]

Martin P. Hellwig schrieb:
> What I don't understand why in the second test, the last boolean is True 
> instead of (what I expect) False.
> Could somebody enlighten me please as this has bitten me before and I am 
> confused by this behavior.

Hint: TEST2.one is not a reference to TEST2.__instance_one.one. When you
alter TEST2.__instance_one.one you don't magically change TEST2.one,
too. Python doesn't have variables like C pointers. Python's copy by
object (or share by object) behavior can be understand as labels. The
label TEST2.one references the same object as TEST2.__instance_one.one
until you change where the label TEST2.__instance_one.one points to.

Christian

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Don't understand behavior; instance form a class in another class' instance

[Martin P. Hellwig]

On 03/25/10 23:41, Christian Heimes wrote:

Martin P. Hellwig schrieb:

What I don't understand why in the second test, the last boolean is True
instead of (what I expect) False.
Could somebody enlighten me please as this has bitten me before and I am
confused by this behavior.


Hint: TEST2.one is not a reference to TEST2.__instance_one.one. When you
alter TEST2.__instance_one.one you don't magically change TEST2.one,
too. Python doesn't have variables like C pointers. Python's copy by
object (or share by object) behavior can be understand as labels. The
label TEST2.one references the same object as TEST2.__instance_one.one
until you change where the label TEST2.__instance_one.one points to.

Christian



Ah okay thanks for the explanation, Am I correct in thinking (please 
correct me if I mangle up the terminology and/or totally are in the 
wrong ballpark) that this is more or less because the label of the first 
class is to an object (boolean with value False)
and the label of the second class does not cascade to the first label 
for looking something up but instead during assignment sees that it is a 
label to an object instead of the object itself thus copies the label 
content instead?


I probably expected classes namespaces to behave in about the same way 
as lists and dictionaries do, don't know where I picked that up.


Thanks again,

--
mph
--
http://mail.python.org/mailman/listinfo/python-list

python logging writes an empty file

[Ovidiu Deac]

Hi,

I have the following situation:

My application uses nosetests to discover&run the unittests. I pass
the log configuration file as --logging-config=logging.conf
Everything works just fine, the logs are printed as required by the
configuration file which makes me happy. I take this as a sign that my
logging.conf is correct

Then in my main script, which starts the production application, I
have this line:
logging.config.fileConfig("logging.conf")

The logging module is configured without errors BUT my output.log is
EMPTY. It's like all the messages are filtered.

If I configure logging like this:
logging.basicConfig(level=logging.INFO,
  format='%(asctime)s %(name)-12s %(levelname)s: %(message)s',
  datefmt='%m-%d %H:%M:%S',
  filename=file,
  filemode='w')
Then the logs are printed ok.

Then I tried this:
file = logging.FileHandler(logFileBasename, 'w')
file.setLevel(logging.INFO)
# set a format which is simpler for console use
formatter = logging.Formatter('%(asctime)s %(name)-12s
%(levelname)-8s %(message)s',)
# tell the handler to use this format
file.setFormatter(formatter)
# add the handler to the root logger
logging.getLogger('').addHandler(file)
logging.getLogger('')

...which also leaves my output file EMPTY.

I'm out of ideas. Can anybody help me with this?

Thanks in advance!
ovidiu

Here is my logging.conf:

[formatters]
keys: detailed,simple

[handlers]
keys: console,file

[loggers]
keys: root

[formatter_simple]
format: %(name)s:%(levelname)s:  %(message)s

[formatter_detailed]
format: %(name)s:%(levelname)s %(module)s:%(lineno)d:  %(message)s

[handler_console]
class: StreamHandler
args: []
formatter: detailed

[handler_file]
class=FileHandler
level=DEBUG
formatter=detailed
args=('output.log', 'w')
filename=output.log
mode=w

[logger_root]
level: INFO
handlers: file
propagate: 1
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Revisiting Generators and Subgenerators

[Cameron Simpson]

On 25Mar2010 14:39, Winston  wrote:
| Here's my proposal again, but hopefully with better formatting so you
| can read it easier.

Having quickly read the Abstract and Motivation, why is this any better
than a pair of threads and a pair of Queue objects? (Aside from
co-routines being more lightweight in terms of machine resources?)

On the flipside, given that generators were recently augumented to
support coroutines I can see your motivation within that framework.

Cheers,
-- 
Cameron Simpson  DoD#743
http://www.cskk.ezoshosting.com/cs/

C makes it easy for you to shoot yourself in the foot.  C++ makes that
harder, but when you do, it blows away your whole leg.
- Bjarne Stroustrup
-- 
http://mail.python.org/mailman/listinfo/python-list

Traversing through Dir()

[Andrej Mitrovic]

I would like to traverse through the entire structure of dir(), and
write it to a file.

Now, if I try to write the contents of dir() to a file (via pickle), I
only get the top layer. So even if there are lists within the returned
list from dir(), they get written as a list of strings to the file.

Basically, I have an embedded and somewhat stripped version of Python.
I would like to find out just how much functionality it has (I have no
documentation for it), so I thought the best way to do that is
traverse thru the dir() call. Any clues as to how I could write the
whole structure to a file? I guess I'll need some kind of recursion
here. :)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Revisiting Generators and Subgenerators

[Winston Wolff]

Coroutines achieve very similar things to threads, but avoid problems resulting 
from the pre-emptive nature of threads. Specifically, a coroutine indicates 
where it will yield to the other coroutine. This avoids lots of problems 
related to synchronization. Also the lightweight aspect is apparently important 
for some simulations when they have many thousands of agents to simulate--this 
number of threads becomes a problem.

-Winston

Winston Wolff
Stratolab - Games for Learning
tel: (646) 827-2242
web: www.stratolab.com

On Mar 25, 2010, at 5:23 PM, Cameron Simpson wrote:

> 
> Having quickly read the Abstract and Motivation, why is this any better
> than a pair of threads and a pair of Queue objects? (Aside from
> co-routines being more lightweight in terms of machine resources?)
> 
> On the flipside, given that generators were recently augumented to
> support coroutines I can see your motivation within that framework.
> 
> Cheers,
> -- 
> Cameron Simpson  DoD#743
> http://www.cskk.ezoshosting.com/cs/
> 
> C makes it easy for you to shoot yourself in the foot.  C++ makes that
> harder, but when you do, it blows away your whole leg.
> - Bjarne Stroustrup

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Represent object type as

[Jason]

On Mar 26, 12:00 am, Bruno Desthuilliers  wrote:
>            attrs['type'] = type(self)
>
> Do the same thing with less work !-)

Ah, silly me :P

>      attrs['__typename__'] = type(self).__name__

That's exactly what I needed — I was not aware of the "__name__"
attribute.

> Warning: won't be very useful if your code still uses old-style classes.

No, all the objects are new-style classes so that's fine.

> Depends on what you do with this dict, DBUS etc. And of your definition
> of "better", of course.

Simplest possible :P But this looks like it, so thanks very much :)

Cheers,
Jason
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Traversing through Dir()

[Shashwat Anand]

have you tried os.walk() ?

On Fri, Mar 26, 2010 at 5:55 AM, Andrej Mitrovic  wrote:

> I would like to traverse through the entire structure of dir(), and
> write it to a file.
>
> Now, if I try to write the contents of dir() to a file (via pickle), I
> only get the top layer. So even if there are lists within the returned
> list from dir(), they get written as a list of strings to the file.
>
> Basically, I have an embedded and somewhat stripped version of Python.
> I would like to find out just how much functionality it has (I have no
> documentation for it), so I thought the best way to do that is
> traverse thru the dir() call. Any clues as to how I could write the
> whole structure to a file? I guess I'll need some kind of recursion
> here. :)
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Automatic import ?

[C. B.]

Hi everyone,

I'm currently coding a C library which provides several modules and
objects.

Let's say that some of these objects are classes called AAA and BBB.
The constructor of AAA needs to get BBB as argument.

So I can run the following code :

from mymodule import AAA
from mymodule import BBB

a = AAA(BBB()))

But, as there is no case where AAA can be used without BBB, I would
like to avoid importing BBB in my Python scripts when I already import
AAA.

For now, I think that reproducing the behavior of the __init__.py file
could be a good way to do this, but how can I code that using only the
C API ?

Are there any other solutions ? Is this kind of importation a good
idea ?

Greetings,
Cyrille Bagard
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Don't understand behavior; instance form a class in another class' instance

[Rhodri James]

On Fri, 26 Mar 2010 00:06:06 -, Martin P. Hellwig  
 wrote:



On 03/25/10 23:41, Christian Heimes wrote:

Martin P. Hellwig schrieb:
What I don't understand why in the second test, the last boolean is  
True

instead of (what I expect) False.
Could somebody enlighten me please as this has bitten me before and I  
am

confused by this behavior.


Hint: TEST2.one is not a reference to TEST2.__instance_one.one. When you
alter TEST2.__instance_one.one you don't magically change TEST2.one,
too. Python doesn't have variables like C pointers. Python's copy by
object (or share by object) behavior can be understand as labels. The
label TEST2.one references the same object as TEST2.__instance_one.one
until you change where the label TEST2.__instance_one.one points to.

Christian



Ah okay thanks for the explanation, Am I correct in thinking (please  
correct me if I mangle up the terminology and/or totally are in the  
wrong ballpark) that this is more or less because the label of the first  
class is to an object (boolean with value False)
and the label of the second class does not cascade to the first label  
for looking something up but instead during assignment sees that it is a  
label to an object instead of the object itself thus copies the label  
content instead?


Pretty much.  In the sense that you're thinking of, every assignment works  
that way, even the initial "TEST1 = One()".  Assignment binds names to  
objects, though you have to be aware that names can be such exotic things  
as "t", "a[15]" or "TEST2.__instance_one.one"


I probably expected classes namespaces to behave in about the same way  
as lists and dictionaries do, don't know where I picked that up.


They do, in fact, which isn't terribly surprising considering that class  
namespaces are implemented with dictionaries.  The distinction you're  
missing is that lists and dictionaries are mutable, while booleans aren't;  
you can change the contents of a dictionary, but you can't change the  
'contents' of a boolean.


--
Rhodri James *-* Wildebeeste Herder to the Masses
--
http://mail.python.org/mailman/listinfo/python-list

Re: Is there any library for indexing binary data?

[甜瓜]

Many thanks for your kind reply. As you mentioned, a sparse array may
be the best choice.
Storing offset rather than payload itself can greatly save memory space.

1e7 queries per second is my ideal aim. But 1e6 must be achieved.
Currently I have implemented 5e6 on one PC (without incremental
indexing and all incoming queries coming from local data stream).
Since the table is very big and responding is time critical, the
finally system will be definitely distributed computing. I hope that
Judy algorithm can simplify indexing, so I can focus on implementing
data persistence and distributed computing affairs.

--
ShenLei

在 2010年3月26日 上午2:55，Irmen de Jong  写道：
> On 25-3-2010 10:55, 甜瓜 wrote:
>> Thank you irmen. I will take a look at pytable.
>> FYI, let me explain the case clearly.
>>
>> Originally, my big data table is simply array of Item:
>> struct Item
>> {
>>  long id;// used as key
>>  BYTE payload[LEN];   // corresponding value with fixed length
>> };
>>
>> All items are stored in one file by using "stdio.h" function:
>>  fwrite(itemarray, sizeof(Item), num_of_items, fp);
>>
>> Note that "id" is randomly unique without any order. To speed up
>> searching  I regrouped / sorted them into two-level hash tables (in
>> the form of files).  I want to employ certain library to help me index
>> this table.
>>
>> Since the table contains about 10^9 items and LEN is about 2KB, it is
>> impossible to hold all data in memory. Furthermore, some new item may
>> be inserted into the array. Therefore incremental indexing feature is
>> needed.
>
> I see, I thought the payload data was small as well. What about this idea:
> Build a hash table where the keys are the id from your Item structs and
> the value is the file seek offset of the Item 'record' in your original
> datafile. (although that might generate values of type long, which take
> more memory than int, so maybe we should use file_offset/sizeof(Item).
> This way you can just keep your original data file (you only have to
> scan it to build the hash table) and you will avoid a lengthy conversion
> process.
>
> If this hashtable still doesn't fit in memory use a sparse array
> implementation of some sort that is more efficient at storing simple
> integers, or just put it into a database solution mentioned in earlier
> responses.
>
> Another thing: I think that your requirement of 1e7 lookups per second
> is a bit steep for any solution where the dataset is not in core memory
> at once though.
>
> Irmen.
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Create a class at run-time

[I V]

On Thu, 25 Mar 2010 15:00:35 -0700, Michel wrote:
> I'm trying to dynamically create a class. What I need is to define a
> class, add methods to it and later instantiate this class. Methods need
> to be bound to the instance though, and that's my problem. Here is what
> I have so far:

I'm not entirely sure what you mean by binding methods to the instance. 
Do you mean you need to dynamically add methods to a specific instance? 
Or that you need to add methods to a class, such that they can be invoked 
on specific instances? For the latter, just do:

TestClass.test_foo = test_foo

For the former, try:

tc = TestClass()
tc.test_foo = types.MethodType(test_foo, tc)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Don't understand behavior; instance form a class in another class' instance

[Martin P. Hellwig]

On 03/26/10 01:10, Rhodri James wrote:



Pretty much. In the sense that you're thinking of, every assignment
works that way, even the initial "TEST1 = One()". Assignment binds names
to objects, though you have to be aware that names can be such exotic
things as "t", "a[15]" or "TEST2.__instance_one.one"


I probably expected classes namespaces to behave in about the same way
as lists and dictionaries do, don't know where I picked that up.


They do, in fact, which isn't terribly surprising considering that class
namespaces are implemented with dictionaries. The distinction you're
missing is that lists and dictionaries are mutable, while booleans
aren't; you can change the contents of a dictionary, but you can't
change the 'contents' of a boolean.



All makes sense now, thanks Rhodri & Christian.

--
mph
--
http://mail.python.org/mailman/listinfo/python-list

Re: Is there any library for indexing binary data?

[John Nagle]

甜瓜 wrote:
> Well, Database is not proper because 1. the table is very big (~10^9
> rows) 2. we should support very fast *simple* query that is to get
> value corresponding to single key (~10^7 queries / second).

   Ah, crypto rainbow tables.

John Nagle
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Revisiting Generators and Subgenerators

[Patrick Maupin]

On Mar 25, 7:31 pm, Winston Wolff  wrote:

(a bunch of stuff about coroutines)

There have been proposals in the past for more full-featured
generators, that would work as general purpose coroutines.  Among
other things, there were issues with exception propagation, and the
design was deliberately simplified to what we have today.  Before
proposing anything in this area you should carefully read PEPs 288,
325, and 342, and all the discussion about those PEPs in the python-
dev archives.

After reading all that, and still being convinced that you have the
greatest thing since sliced bread (and that you REALLY understand all
the concerns about exceptions and other things), you need to update
your document to address all the concerns raised in the discussions on
those PEPs, put on your asbestos suit (modern asbestos-free
replacements never work as advertised), and then re-post your
document.

Personally, I am very interested in co-routines, but I have very
little time right now, and am not at all interested in reading a
proposal from somebody who doesn't know the full history of how
generators got to be the way they are (the lack of coroutines is not
an accidental oversight).  I suspect I am not alone in this opinion,
so there is probably some interest in a realistic proposal, but
perhaps also some skepticism about whether a realistic proposal can
actually be engineered...

Best regards and good luck!
Pat
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Automatic import ?

[Steven D'Aprano]

On Thu, 25 Mar 2010 18:03:58 -0700, C. B. wrote:

> Hi everyone,
> 
> I'm currently coding a C library which provides several modules and
> objects.
> 
> Let's say that some of these objects are classes called AAA and BBB. The
> constructor of AAA needs to get BBB as argument.
> 
> So I can run the following code :
> 
> from mymodule import AAA
> from mymodule import BBB
> 
> a = AAA(BBB()))
> 
> But, as there is no case where AAA can be used without BBB, I would like
> to avoid importing BBB in my Python scripts when I already import AAA.

Since AAA must take an argument of BBB, then give it a default:

# in mymodule
def AAA(arg=BBB()):
...

# in another module
from mymodule import AAA
a = AAA()


Or do this:

from mymodule import AAA, BBB
a = AAA(BBB())




> For now, I think that reproducing the behavior of the __init__.py file
> could be a good way to do this, but how can I code that using only the C
> API ?

What is the behaviour of the __init__.py file?



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Revisiting Generators and Subgenerators

[Stefan Behnel]

Patrick Maupin, 26.03.2010 04:30:

... and then re-post your document.


... preferably to the python-ideas mailing list. Although it seems to me 
that this is something that could be explored as a library first - which 
usually means that people will tell you exactly that on python-ideas and 
ask you to come back with an implementation that has proven to be useful in 
practice.


Stefan

--
http://mail.python.org/mailman/listinfo/python-list

Question regarding curses and text-input methods

[Harishankar]

I am trying to use Python curses.textpad to get (preferably UTF-8 but
not needed) input from the user in a python curses application, but I
realize that it is extremely limited and doesn't allow me to specify an
existing text to edit. Also it doesn't seem to provide scrolling
abilities on the window.

Is this at all possible to get a large chunk of text using curses
module like a GUI textbox or should I look at alternatives like newt?

I prefer curses because it's simple and very flexible, but I am lost
for ideas on this. Any ideas which doesn't involve throwing away the
whole curses interface would be appreciated.

-- 
V.Harishankar

http://literaryforums.org
http://harishankar.org
-- 
http://mail.python.org/mailman/listinfo/python-list