Re: Python dot-equals (syntax proposal)

[Lie Ryan]

On 05/02/10 10:58, Steven D'Aprano wrote:
> On Sun, 02 May 2010 05:08:53 +1000, Lie Ryan wrote:
> 
>> > On 05/01/10 11:16, Steven D'Aprano wrote:
>>> >> On Fri, 30 Apr 2010 12:34:34 -0400, D'Arcy J.M. Cain wrote:
>>> >> 
>>> >> In practice though, I think that's a difference that makes no
>>> >> difference. It walks like an operator, it swims like an operator, and
>>> >> it quacks like an operator.
>>> >> 
>>> >> 
>> > Nope it's not. A full-time operator in python have a reflected version
>> > (e.g. __radd__), which dot does not have. 
> What are the reflected versions of __eq__ and __ne__ (binary == and != 
> operators)?

Python do not have them now, but it does make sense if python have
them[1]. OTOH, given current python's language semantic, __rgetattr__
doesn't make any sense; adding __rgetattr__ would require a quite
fundamental change to the language's semantic, primarily how attribute
resolution works.

[1] though they'd probably be dismissed as bad idea since equality and
inequality are supposed to be a symmetric relation; reflected
(in)equality makes it way too easy to break that premise

> And __neg__, __pos__ and __inv__ (for the unary - + and ~ operators)?
> 
> And the three-argument form of __pow__ for power(1, 2, x)?

I know you're a famed nitpicker, but don't be silly, reflected operator,
by definition, only makes sense for binary operator.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyjamas 0.7 released

[Wolfgang Strobl]

lkcl :

> at least _some_ input would be good!  the knowledge doesn't have to
>be there: just the bugreports saying "there's a problem and here's
>exactly how you reproduce it" would be a start!

>> So please make it simpler for more people to help.

> ... how?? there's a bugtracker, wiki, svn repository, over 30
>examples and a developer list.  the code really _is_ very very small
>(the UI widget set, with around 75 widgets, minus the license header
>text is only around 4,000 lines _total_, making it very very simple
>and very very easy for people to get used to).  suggestions welcome!

Well, the bunch of programming languages and APIs I collected over the
years is large enough already. These days I prefer to stay with python
and c, spiced with an occasional domain specific language. That's why I
was attracted by pyjamas, to begin with!.  If I'd like to program using
Eclipse and Java or fool around with JavaScript, I'd do just that. But I
don't.  IMHO, that ist a general problem of translation tools - they
attract the wrong people, from the point of view a developer who looks
for people sharing some of the workload. :-)

So, Luke, I can only answer your question from the point of view of
somebody who is mostly a potentional consumer of your work, and most
problably not another developer. If you want to delegate some work you'd
like not to do yourself (for example because you prefer designing and
coding to testing and reorganizing and polishing the docs), than you
have at least to _define_ those pieces and to monitor progress. 

>
>> Other people have
>> other projects, where they invest some of their own time and money in.
>
> yes. been there. didn't receive any return on investment. did some
>projects. received about 25% of required money to pay bills.  need
>£15k pa; receiving approximately £7-8k.
>
>> >>  I didn't look at
>> >> every example again, but AFAIK, this didn't change with 0.7. (Tried with
>> >> Python 2.6.4/5 on WinXP/7 with Firefox and Chrome, if that matters).
>>
>> > then please issue bugreports for as many as you feel comfortable
>> >with / have time for.  

Well, ok. I put my notes in a Google chart, see
http://spreadsheets.google.com/pub?key=0Au5LhqNQyrfCdGpDbTVjZFJwSERzVUFXVlg5bWl2enc

I had to write a short patch against compile.py (see issue 397) in order
to make it compile the showcase examples on Windows.

In addition, I've tried to create Selenium tests for automating the time
consuming job of checking all those examples, using Selenium IDE in
Firefox. I was my first experience using this against Ajax apps. The
results are somewhat mixed. Playing a round of "lightout" was a breeze,
but so far I hadn't much luck in driving the showcase example(s). I
didn't try very hard, though, because I ran out of time, as I do now.  

-- 
Wir danken für die Beachtung aller Sicherheitsbestimmungen
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python dot-equals (syntax proposal)

[Terry Reedy]

On 5/2/2010 1:05 AM, Alf P. Steinbach wrote:

On 02.05.2010 06:06, * Aahz:



and sometimes
they rebind the original target to the same object.


At the Python level that seems to be an undetectable null-operation.


If you try t=(1,2,3); t[1]+=3, if very much matters that a rebind occurs.


Granted one could see something going on in a machine code or byte code
debugger. But making that distinction (doing nothing versus
self-assignment) at the Python level seems, to me, to be meaningless.


Please do not confuse things. Augmented *assignment* must be understood 
as assignment. Failure to do so leads (and has lead) newbies into 
confusion, and puzzled posts on this list.


Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python dot-equals (syntax proposal)

[Steven D'Aprano]

On Sun, 02 May 2010 16:28:28 +1000, Lie Ryan wrote:

> On 05/02/10 10:58, Steven D'Aprano wrote:
>>> > And Python's object system
>>> > makes it that the argument to __getattr__ is always a string even
>>> > though there might be a valid variable that corresponds to it:
>> That is nothing to do with the object system, it is related to the
>> semantics of Python syntax. a.b doesn't mean "apply the binary dot
>> operator to arguments a and b". It is syntactic sugar for "look for an
>> attribute named 'b' on object a". As such, the operands that
>> __getattr__ receives are the object a and the *name* b (implemented as
>> a string).
> 
> You just described *exactly* the reason why dot is not, and cannot be an
> operator.

This would be relevant if I said it was an operator. I did not. I said it 
was a de facto operator that behaves similarly enough to operators as to 
make it reasonable to talk about "dot operator". I still stand by that.

That doesn't imply that Python's implementation of a.b has to be 
identical in every last detail to Python's implementation of a+b, because 
it clearly isn't. But there are sufficient similarities to justify saying 
that it duck-types as an operator. This isn't meant to be a vigorous 
statement of fact in the same manner than "CPython implements lists as 
arrays of pointers" is a statement of fact. It's meant to be a hand-wavy 
"dot act kinda-sorta like an operator" manner.

I'm sorry if I failed to make that clear enough. I thought that 
explicitly stating that it wasn't a real operator would be sufficient.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python dot-equals (syntax proposal)

[Steven D'Aprano]

On Sun, 02 May 2010 17:09:36 +1000, Lie Ryan wrote:

> On 05/02/10 10:58, Steven D'Aprano wrote:
>> On Sun, 02 May 2010 05:08:53 +1000, Lie Ryan wrote:
>> 
>>> > On 05/01/10 11:16, Steven D'Aprano wrote:
 >> On Fri, 30 Apr 2010 12:34:34 -0400, D'Arcy J.M. Cain wrote:
 >> 
 >> In practice though, I think that's a difference that makes no
 >> difference. It walks like an operator, it swims like an operator,
 >> and it quacks like an operator.
 >> 
 >> 
>>> > Nope it's not. A full-time operator in python have a reflected
>>> > version (e.g. __radd__), which dot does not have.
>> What are the reflected versions of __eq__ and __ne__ (binary == and !=
>> operators)?
> 
> Python do not have them now, but it does make sense if python have
> them[1]. OTOH, given current python's language semantic, __rgetattr__
> doesn't make any sense; adding __rgetattr__ would require a quite
> fundamental change to the language's semantic, primarily how attribute
> resolution works.
> 
> [1] though they'd probably be dismissed as bad idea since equality and
> inequality are supposed to be a symmetric relation; reflected
> (in)equality makes it way too easy to break that premise
> 
>> And __neg__, __pos__ and __inv__ (for the unary - + and ~ operators)?
>> 
>> And the three-argument form of __pow__ for power(1, 2, x)?
> 
> I know you're a famed nitpicker, but don't be silly, reflected operator,
> by definition, only makes sense for binary operator.

Binary operators aren't the only kind of operator, and you claimed that:

"A full-time operator in python have a reflected version".

But there are full-time operators that don't have reflected versions, so 
your claim is just *wrong*. It would still be wrong even if you had 
limited yourself to binary operators.

I have agreed with you that there are useful things people might want to 
do (e.g. function composition) that you can't do because the "dot 
operator" isn't a *real* operator with exactly the same semantics as 
"plus operator", "multiply operator" and friends. I think we're in 
violent agreement, and merely disagreeing over semantics. There's no need 
to continue arguing against a position I haven't actually taken :)


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sometype.__new__ and C subclasses

[Carl Banks]

On May 1, 11:03 pm, James Porter  wrote:
> I've been trying to write a Python C extension module that uses NumPy
> and has a subtype of numpy.ndarray written in C. However, I've run into
> a snag: calling numpy.ndarray.__new__(mysubtype, ...) triggers an
> exception in the bowels of Python (this is necessary for a handful of
> NumPy features). I'm posting to this list to try to figure out why this
> exception exists in the first place, and what (if anything) I can do to
> work around it.
>
> The exception in question happens in Objects/typeobject.c in
> tp_new_wrapper. Here's the comment for the block:
>
>         /* Check that the use doesn't do something silly and unsafe like
>            object.__new__(dict).  To do this, we check that the
>            most derived base that's not a heap type is this type. */
>
> The code has the end effect that basetype.__new__(subtype, ...) fails
> whenever subtype is a statically-defined type (i.e. a normal C extension
> type object). Why is this necessary in general? I can see why it might
> be bad for a limited number of core Python types, but it seems
> unnecessarily limiting for Python C extensions.

Why don't you use mysubtype.__new__(mysubtype,...)?

If you wrote mysubtype in C, and defined a different tp_new than
ndarray, then this exception will trigger.  And it ought to; you don't
want to use ndarray's tp_new to create an object of your subclass, if
you've defined a different tp_new.


> On a more practical note, is there a way (short of rewriting the subtype
> in Python) to work around this? It seems that I could call the type
> metaclass to create a heap type in C, but I'm not sure of all the
> implications of that.

It should work if you use mysubtype.__new__(mysubtype,...).

If it doesn't do what you want, then there's probably something wrong
with the way you subclassed ndarray.


Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PyObject_SetAttrString - doesn't set instance attribute

[Carl Banks]

On May 1, 4:04 am, Jason  wrote:
> I'm having a bit of trouble with C/Python bindings. Particularly,
> trying to set an instance variable from C when the object is
> initialised using PyObject_SetAttrString, but nothing seems to happen.
> The C initialisation code is:
>
> static void
> nautilus_python_object_instance_init (NautilusPythonObject *object)
> {
>
>         fprintf(stderr, "nautilus_python_object_instance_init called\n");
>
>         NautilusPythonObjectClass *class;
>         debug_enter();
>
>         class = (NautilusPythonObjectClass*)(((GTypeInstance*)object)-
>
> >g_class);
>
>         object->instance = PyObject_CallObject(class->type, NULL);
>
>         PyObject* test_int = PyInt_FromLong(42);
>
>         if (object->instance == NULL)
>         {
>                 PyErr_Print();
>         }
>         else
>         {
>                 fprintf(stderr, "Setting magic parameter\n");
>                 fprintf(stderr, "From C: ");
>                 PyObject_Print(object->instance, stderr, 0);
>                 fprintf(stderr, "\n");
>                 int retval = PyObject_SetAttrString(object->instance,
> "super_happy_magic", test_int);
>                 fprintf(stderr, "Result: %i\n", retval);
>         }
>
>         Py_DECREF(test_int);
>
> }

Not sure what you're doing here.  It looks like you are being passed
an object of a given type, then you get the type object, call it to
create another object of that type, and assign it to object->instance.


> ...and the Python module contains:
>
> class MenuProviderTest(nautilus.MenuProvider):
>
>     def __init__(self):
>         print "From Python: %s" % self
>
>         try:
>             print getattr(self, "super_happy_magic")
>         except AttributeError:
>             print "Didn't work!"
>
> When the MenuProviderTest is created, the output is:
>
> nautilus_python_object_instance_init called
> Setting magic parameter
> From C: 
> Result: 0
> From Python: 
> Didn't work!
>
> (I've tried getattr and self.super_happy_magic, with the same effect.)
>
> Where am I going wrong?

You are assigning the attirbute the the object that the C code refers
to as "object->instance", but it seems that in the Python snippet you
are calling getattr on the object that the C code refers to as
"object".


Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

help error : Failed to build these modules: _dbm

[michel parker]

Hi,
When i make python 3.1 on my ubuntu 9.10 i get following error :

Failed to build these modules:
_dbm 
Please help me.
I have done :
apt-get build-dep python2.5

but to no avail.
Cheers

  
_
Hotmail: Free, trusted and rich email service.
https://signup.live.com/signup.aspx?id=60969-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python dot-equals (syntax proposal)

[Steven D'Aprano]

On Sun, 02 May 2010 04:04:11 -0400, Terry Reedy wrote:

> On 5/2/2010 1:05 AM, Alf P. Steinbach wrote:
>> On 02.05.2010 06:06, * Aahz:
> 
>>> and sometimes
>>> they rebind the original target to the same object.
>>
>> At the Python level that seems to be an undetectable null-operation.
> 
> If you try t=(1,2,3); t[1]+=3, if very much matters that a rebind
> occurs.
> 
>> Granted one could see something going on in a machine code or byte code
>> debugger. But making that distinction (doing nothing versus
>> self-assignment) at the Python level seems, to me, to be meaningless.
> 
> Please do not confuse things. Augmented *assignment* must be understood
> as assignment. Failure to do so leads (and has lead) newbies into
> confusion, and puzzled posts on this list.

I think that if you think *only* about Python's standard namespaces, self-
assignment is more or less a no-op. I can't think of any way that x = x 
could do anything other than use CPU cycles, *if* you limit yourself to 
the standard global or function local namespaces.

But if you think about custom namespace types, and item assignment (e.g. 
the example you gave with a tuple), the situation becomes different. 
Here's a nice example, using Python 3.1:

>>> class B(A):  # class A defined elsewhere -- see below.
... x = 1
... x = x
...
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 3, in B
  File "", line 4, in __setitem__
TypeError: can't rebind constants

Thus proving that self-assignment is not necessarily a no-op. How did I 
make that work? It takes a custom dict and a bit of metaclass magic:


class ConstantNamespace(dict):
def __setitem__(self, key, value):
if key in self:
raise TypeError("can't rebind constants")
super(ConstantNamespace, self).__setitem__(key, value)

class WriteOnceClass(type):
@classmethod
def __prepare__(metacls, name, bases):
return ConstantNamespace()
def __new__(cls, name, bases, classdict):
return type.__new__(cls, name, bases, classdict)

class A(metaclass=WriteOnceClass):
pass



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PyObject_SetAttrString - doesn't set instance attribute

[Jason]

On May 2, 5:52 pm, Carl Banks  wrote:
> Not sure what you're doing here.  It looks like you are being passed
> an object of a given type, then you get the type object, call it to
> create another object of that type, and assign it to object->instance.

Sorry, I should have noted that the "NautilusPythonObject" type in the
code is a struct defined as:

struct _NautilusPythonObject {
  GObject parent_slot;
  PyObject *instance;
};

> You are assigning the attirbute the the object that the C code refers
> to as "object->instance", but it seems that in the Python snippet you
> are calling getattr on the object that the C code refers to as
> "object".

object->instance is the PyObject, and I gathered that it was the
correct thing to assign to from the fact that the address is identical
as seen from C and Python.

— Jason
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: help error : Failed to build these modules: _dbm

[Kushal Kumaran]

On Sun, May 2, 2010 at 3:30 PM, michel parker  wrote:
> Hi,
> When i make python 3.1 on my ubuntu 9.10 i get following error :
>
> Failed to build these modules:
> _dbm
> Please help me.
> I have done :
>
> apt-get build-dep python2.5
>
> but to no avail.
> Cheers
>
>

Probably the packages libdb4.7-dev (if you want berkeley db) or
libgdbm-dev (if you want gnu dbm).

ISTR there was a thread about this just a couple of days ago.

-- 
regards,
kushal
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python dot-equals (syntax proposal)

[Albert van der Horst]

In article ,
Jean-Michel Pichavant   wrote:
>Jabapyth wrote:
>> At least a few times a day I wish python had the following shortcut
>> syntax:
>>
>> vbl.=func(args)
>>
>> this would be equivalent to
>>
>> vbl = vbl.func(args)
>>
>> example:
>>
>> foo = "Hello world"
>> foo.=split(" ")
>> print foo
>> # ['Hello', 'world']
>>
>> and I guess you could generalize this to
>>
>> vbl.=[some text]
>> #
>> vbl = vbl.[some text]
>>
>> e.g.
>>
>> temp.=children[0]
>> # temp = temp.children[0]
>>
>> thoughts?
>>
>Useless if you use meaningful names for your variables & attributes.
>
>It may happen that one object attribute refer to an object of the same
>type, but it is quite rare that both can share the same name anyway.
>
>Possible use cases:
>
>1/
>car = Car()
>car = car.wheel # ???
>
>2/
>wheel = Car() # ???
>wheel = wheel.wheel # ???
>
>3/
>currentCar = Car()
>currentCar = currentCar.nextCar
>
>The syntax you prose will be applicable on very little assignements (use
>case 3). I'm not sure it's worth it.

Note how related it is to the requirement to have a _radd_ operator.

It amounts to the argument that
a op= b
requires that a and b have somewhat "similar" "type", or that
the "type" of a doesn't really change as a result from the operation.

This is IMHO an argument against the .= pseudo-operator.

>
>JM

Groetjes Albert

--
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
alb...@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: matching strings in a large set of strings

[Albert van der Horst]

In article <877hnpjtdw@rudin.co.uk>,
Paul Rudin   wrote:
>"Karin Lagesen"  writes:
>
>> Hello.
>>
>> I have approx 83 million strings, all 14 characters long. I need to be
>> able to take another string and find out whether this one is present
>> within the 83 million strings.
>>
>> Now, I have tried storing these strings as a list, a set and a dictionary.
>> I know that finding things in a set and a dictionary is very much faster
>> than working with a list, so I tried those first. However, I run out of
>> memory building both the set and the dictionary, so what I seem to be left
>> with is the list, and using the in method.
>>
>> I imagine that there should be a faster/better way than this?
>>
>
>Shouldn't a set with 83 million 14 character strings be fine in memory
>on a stock PC these days? I suppose if it's low on ram you might start
>swapping which will kill performance. Perhaps the method you're using to
>build the data structures creates lots of garbage? How much ram do you
>have and how much memory does the python process use as it builds your
>data structures?

And if this is practical there should be no swapping problems,
as the working set will be a small fraction of the data used.


>
>There are other algorithms you can use that have better theoretical
>performance than using bisect for this particular problem, but you get
>into trade offs between executing things in python as opposed to C if
>you start to hand code things in python.

There are a lot of possibility, but they depend a great deal on
secondary conditions, like how often the 83 M dataset changes.

Groetjes Albert

--
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
alb...@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

-- 
http://mail.python.org/mailman/listinfo/python-list

RE: help error : Failed to build these modules: _dbm

[michel parker]

hi,
yup problem solved module was missing  libgdbm-dev
> From: kushal.kumaran+pyt...@gmail.com
> Date: Sun, 2 May 2010 16:38:28 +0530
> Subject: Re: help error : Failed to build these modules: _dbm
> To: michelpar...@live.com
> CC: python-list@python.org
> 
> On Sun, May 2, 2010 at 3:30 PM, michel parker  wrote:
> > Hi,
> > When i make python 3.1 on my ubuntu 9.10 i get following error :
> >
> > Failed to build these modules:
> > _dbm
> > Please help me.
> > I have done :
> >
> > apt-get build-dep python2.5
> >
> > but to no avail.
> > Cheers
> >
> >
> 
> Probably the packages libdb4.7-dev (if you want berkeley db) or
> libgdbm-dev (if you want gnu dbm).
> 
> ISTR there was a thread about this just a couple of days ago.
> 
> -- 
> regards,
> kushal
  
_
Hotmail: Trusted email with powerful SPAM protection.
https://signup.live.com/signup.aspx?id=60969-- 
http://mail.python.org/mailman/listinfo/python-list

help req setting python executable

[michel parker]

hi,
i have just installed python3.1 with opt=g option.
but when i set python executable in wingide to usr/local/bin/python3.1 it says :
Some values are invalid:

- Python executable 'usr/local/bin/python3.1' is not in path

Please correct the values and try again.

Please help. What is going to be correct pythonpath for me.
i have installed python using standard sudo make install method.
Cheers
  
_
Hotmail: Powerful Free email with security by Microsoft.
https://signup.live.com/signup.aspx?id=60969-- 
http://mail.python.org/mailman/listinfo/python-list

Bug in Python set

[dmitrey]

Python 2.6.5 r265:79063
>>>set().update(set()) is None
True
while I expect result of update to be set.
Also, result of
set().add(None)
is None while I expect it to be set with element None (or, maybe, it
should be empty set?)

Regards, D.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: recommended way to insert data into a one to many relationship using python

[Peter Otten]

Peter Otten wrote:

If you create indices for floors (and rooms)

cur.execute("""create unique index room_index on rooms (fid, number);""")
cur.execute("""create unique index floor_index on floors (floor);""")

the addition of missing rows can be simplified to

missing = c2.execute("""select distinct floor from new_employees;""")
cur.executemany("insert or ignore into floors (floor) values (?)", missing)

etc.

Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Bug in Python set

[Aahz]

In article <0bd314a8-db65-43f1-a999-521e2ed71...@n15g2000yqf.googlegroups.com>,
dmitrey   wrote:
>
>Python 2.6.5 r265:79063
set().update(set()) is None
>True
>while I expect result of update to be set.
>Also, result of
>set().add(None)
>is None while I expect it to be set with element None (or, maybe, it
>should be empty set?)

Why are you assuming that your expectations are correct?  Generally
speaking, Python methods that mutate do *not* return the original object,
precisely to make sure you don't make stupid mistakes.

You should probably read this:

http://www.catb.org/~esr/faqs/smart-questions.html
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Bug in Python set

[Steven D'Aprano]

On Sun, 02 May 2010 05:11:40 -0700, dmitrey wrote:

> Python 2.6.5 r265:79063
set().update(set()) is None
> True
> while I expect result of update to be set.

Change your expectations. Generally, methods which modify the object 
rather than creating a new one return None.

>>> s = set([1,2,3])
>>> s.update(set([3, 4, 5]))
>>> s
{1, 2, 3, 4, 5}


> Also, result of set().add(None) is None while I expect it to be set
> with element None (or, maybe, it should be empty set?)

>>> s = set()
>>> s.add(None)
>>> s
{None}


Python sets have been used by tens of thousands of programmers for many 
years now. Which do you think is more likely?

(1) Not one person before you noticed that something as fundamental as 
adding an item to a set is buggy;

or

(2) You have misunderstood what is happening?



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

List size versus list allocation

[Steven D'Aprano]

Python lists are over-allocated: whenever they need to be resized, they 
are made a little bit larger than necessary so that appends will be fast.

See:

http://code.python.org/hg/trunk/file/e9d930f8b8ff/Objects/listobject.c

I'm interested in gathering some statistics on this, and to do so I need 
a way of measuring the list's logical size versus its actual size. The 
first is easy: it's just len(list). Is there some way of getting the 
allocated size of the list?



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Fast Efficient way to transfer an object to another list

[Hans Mulder]

Francesco Bochicchio wrote:


Anyway i think that list.extract( old_list, predicate ) -> new_list
would be a nice addition to the standard library


You could use filter( predicate, old_list ) -> new_list

-- HansM



--
http://mail.python.org/mailman/listinfo/python-list

Re: help req setting python executable

[D'Arcy J.M. Cain]

On Sun, 2 May 2010 12:59:35 +0100
michel parker  wrote:
> 
> hi,
> i have just installed python3.1 with opt=g option.
> but when i set python executable in wingide to usr/local/bin/python3.1 it 
> says :

I don't know anything about wingide but I think your problem is simply
a missing slash.

/usr/local/bin/python3.1

-- 
D'Arcy J.M. Cain  |  Democracy is three wolves
http://www.druid.net/darcy/|  and a sheep voting on
+1 416 425 1212 (DoD#0082)(eNTP)   |  what's for dinner.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: List size versus list allocation

[Peter Otten]

Steven D'Aprano wrote:

> Python lists are over-allocated: whenever they need to be resized, they
> are made a little bit larger than necessary so that appends will be fast.
> 
> See:
> 
> http://code.python.org/hg/trunk/file/e9d930f8b8ff/Objects/listobject.c
> 
> I'm interested in gathering some statistics on this, and to do so I need
> a way of measuring the list's logical size versus its actual size. The
> first is easy: it's just len(list). Is there some way of getting the
> allocated size of the list?

With some brute force guesswork...

>>> a = [None] * 42
>>> for i in range(10):
... print i, ctypes.c_ulong.from_address(id(a)+i*8)
...
0 c_ulong(1L)
1 c_ulong(8488896L)
2 c_ulong(42L)
3 c_ulong(37432768L)
4 c_ulong(42L)
5 c_ulong(1L)
6 c_ulong(8510496L)
7 c_ulong(32L)
8 c_ulong(18446744073709551615L)
9 c_ulong(8935143189711421440L)
>>> a = []
>>> for i in range(42): a.append(None)
...
>>> for i in range(10):
... print i, ctypes.c_ulong.from_address(id(a)+i*8)
...
0 c_ulong(1L)
1 c_ulong(8488896L)
2 c_ulong(42L)
3 c_ulong(40136160L)
4 c_ulong(46L)
5 c_ulong(0L)
6 c_ulong(0L)
7 c_ulong(0L)
8 c_ulong(0L)
9 c_ulong(0L)

... you can see that the size sits at offset 4*sizeof(long) on my 64-bit 
Python. With that information we can take a look at a list's overallocation 
strategy:

>>> a = []
>>> for i in range(20):
... print i, ctypes.c_ulong.from_address(id(a)+4*8)
... a.append(None)
...
0 c_ulong(0L)
1 c_ulong(4L)
2 c_ulong(4L)
3 c_ulong(4L)
4 c_ulong(4L)
5 c_ulong(8L)
6 c_ulong(8L)
7 c_ulong(8L)
8 c_ulong(8L)
9 c_ulong(16L)
10 c_ulong(16L)
11 c_ulong(16L)
12 c_ulong(16L)
13 c_ulong(16L)
14 c_ulong(16L)
15 c_ulong(16L)
16 c_ulong(16L)
17 c_ulong(25L)
18 c_ulong(25L)
19 c_ulong(25L)

Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: List size versus list allocation

[Christian Heimes]

I'm interested in gathering some statistics on this, and to do so I need
a way of measuring the list's logical size versus its actual size. The
first is easy: it's just len(list). Is there some way of getting the
allocated size of the list?


With Python 2.6 and newer you can use sys.getsizeof() to get the actual 
size of the list object:



import sys, struct
sys.getsizeof([])

72

a = [1,2,3]
b = []
b.append(1)
b.append(2)
b.append(3)
sys.getsizeof(a)

96

sys.getsizeof(b)

104

def slots(lst):
... return (sys.getsizeof(lst) - sys.getsizeof([])) / 
struct.calcsize("P")

...

slots(b)

4

slots(a)

3

Christian

--
http://mail.python.org/mailman/listinfo/python-list

Re: List size versus list allocation

[Peter Otten]

Christian Heimes wrote:

 def slots(lst):
> ... return (sys.getsizeof(lst) - sys.getsizeof([])) /

D'oh!

Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

condition and True or False

[Paul McGuire]

While sifting through some code looking for old "x and y or z" code
that might better be coded using "y if x else z", I came across this
puzzler:

x =  and True or False

What is "and True or False" adding to this picture?  The boolean
expression part is already evaluating to a boolean, so I don't
understand why a code author would feel compelled to beat this one
over the head with the additional "and True or False".

I did a little code Googling and found a few other Python instances of
this, but also many Lua instances.  I'm not that familiar with Lua, is
this a practice that one who uses Lua frequently might carry over to
Python, not realizing that the added "and True or False" is redundant?

Other theories?

-- Paul
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sometype.__new__ and C subclasses

[James Porter]

On 5/2/2010 4:34 AM, Carl Banks wrote:

Why don't you use mysubtype.__new__(mysubtype,...)?

If you wrote mysubtype in C, and defined a different tp_new than
ndarray, then this exception will trigger.  And it ought to; you don't
want to use ndarray's tp_new to create an object of your subclass, if
you've defined a different tp_new.


Unfortunately, I can't do that, since that call is in NumPy itself and 
it's part of their "standard" way of making instances of subclasses of 
ndarray. Functions like numpy.zeros_like use ndarray.__new__(subtype, 
...) to create new arrays based on the shape of other arrays.


The Python version of the subclass is shown here: 
, 
and I'm trying to write something pretty similar in C. I'm trying to 
stay in C since everything else is in C, so it's easier to stay in C 
then to jump back and forth all the time.


Maybe the real answer to this question is "NumPy is doing it wrong" and 
I should be on their list; still, it seems strange that the behavior is 
different between Python and C.


- Jim

--
http://mail.python.org/mailman/listinfo/python-list

Python debuggers with sys.settrace()

[Sarah Mount]

This is a bit of an odd question, but is there any way for a Python
debugger to suppress I/O generated by the program which is being
debugged? I guess an "obvious" thing to do would be to replace core
parts of the standard library and change any relevant imports in the
locals and globals dicts to fake ones which don't generate I/O, but
this seems brittle as the standard library will change over time. Is
it possible to modify the byte-compiled code in each stack frame? Or
is there a simpler way to do this?

Many thanks,

Sarah

-- 
Sarah Mount, Senior Lecturer, University of Wolverhampton
website:  http://www.snim2.org/
twitter: @snim2
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: condition and True or False

[Peter Otten]

Paul McGuire wrote:

> While sifting through some code looking for old "x and y or z" code
> that might better be coded using "y if x else z", I came across this
> puzzler:
> 
> x =  and True or False
> 
> What is "and True or False" adding to this picture?  The boolean
> expression part is already evaluating to a boolean, so I don't
> understand why a code author would feel compelled to beat this one
> over the head with the additional "and True or False".
> 
> I did a little code Googling and found a few other Python instances of
> this, but also many Lua instances.  I'm not that familiar with Lua, is
> this a practice that one who uses Lua frequently might carry over to
> Python, not realizing that the added "and True or False" is redundant?
> 
> Other theories?

If it were e. g.

def f():
big_beast = list(range(10**100))
return big_beast and True or False
x = f()

it would prevent that a big_beast reference becomes visible outside the 
function and allow for immediate release of its memory.

Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: assigning multi-line strings to variables

[Sibylle Koczian]

goldtech schrieb:

Thank you to posters for help to my question. Seems I had trouble with
triple quotes strings in the PythonWin shell. But using the Idle shell
things work as expected. But this is probably another issue...any way,
w/Idle's shell I got the "action" regarding multiline strings I
expected.


What PythonWin version? I remember having had such trouble years ago, 
probably Python 1.something.


Greetings
Sibylle
--
http://mail.python.org/mailman/listinfo/python-list

Re: condition and True or False

[Vito 'ZeD' De Tullio]

Peter Otten wrote:

> def f():
> big_beast = list(range(10**100))
> return big_beast and True or False
> x = f()
> 
> it would prevent that a big_beast reference becomes visible outside the
> function and allow for immediate release of its memory.

what's wrong in bool(big_beast)?

-- 
By ZeD

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sometype.__new__ and C subclasses

[Robert Kern]

On 2010-05-02 12:48 , James Porter wrote:

On 5/2/2010 4:34 AM, Carl Banks wrote:

Why don't you use mysubtype.__new__(mysubtype,...)?

If you wrote mysubtype in C, and defined a different tp_new than
ndarray, then this exception will trigger. And it ought to; you don't
want to use ndarray's tp_new to create an object of your subclass, if
you've defined a different tp_new.


Unfortunately, I can't do that, since that call is in NumPy itself and
it's part of their "standard" way of making instances of subclasses of
ndarray. Functions like numpy.zeros_like use ndarray.__new__(subtype,
...) to create new arrays based on the shape of other arrays.

The Python version of the subclass is shown here:
,
and I'm trying to write something pretty similar in C. I'm trying to
stay in C since everything else is in C, so it's easier to stay in C
then to jump back and forth all the time.

Maybe the real answer to this question is "NumPy is doing it wrong" and
I should be on their list; still, it seems strange that the behavior is
different between Python and C.


Perhaps things would be clearer if you could post the C code that you've written 
that fails. So far, you've only alluded at what you are doing using 
Python-syntax examples.


--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list

Re: recommended way to insert data into a one to many relationship using python

[Bryan]

Wolfgang Meiners wrote:
> one to many relationships are fairly common, i think. So there should be
> a recommended way to insert data into such a relation using python.
>
> Given the following programm, what is the recommended way to insert the
> list of NewEmployees to the database?
>
> 
> # !python
> # -*- coding: utf-8 -*-
>
> import sqlite3
>
> con = sqlite3.connect(":memory:")
> cur = con.cursor()
>
> cur.execute("""create table employees(
>     eid integer primary key autoincrement,
>     name text not null,
>     rid integer references rooms(rid))""")
>
> cur.execute("""create table rooms(
>     rid integer primary key autoincrement,
>     number integer,
>     fid integer references floors(fid))""")
>
> cur.execute("""create table floors(
>     fid integer primary key autoincrement,
>     floor text not null)""")
>
[...]
>
> NewEmployees  =[]
> NewEmployees.append({'name': 'George', 'room': 89, 'floor': 'third floor'})
> NewEmployees.append({'name': 'Ellen', 'room': 21, 'floor': 'first floor'})
>

For that kind of insert to be well-defined, the pair (floor,
room_number) must uniquely identify a room. When natural keys like
that are availabe, they're the obvious choice for primary keys in the
database schema. I suggested getting rid of fid and rid, as in:


schema = """
CREATE TABLE floors (
floor TEXT PRIMARY KEY
);

CREATE TABLE rooms (
floor TEXT REFERENCES floors,
number INTEGER,
PRIMARY KEY (floor, number)
);

CREATE TABLE employees (
eid INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT,
floor TEXT,
room_number INTEGER,
FOREIGN KEY (floor, room_number) REFERENCES rooms
)
"""

con = sqlite3.connect(":memory:")
for cmd in schema.split(';'):
con.execute(cmd)
con.close()



--
--Bryan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Bug in Python set

[Terry Reedy]

On 5/2/2010 8:11 AM, dmitrey wrote:

Python 2.6.5 r265:79063

set().update(set()) is None

True
while I expect result of update to be set.
Also, result of
set().add(None)
is None while I expect it to be set with element None (or, maybe, it
should be empty set?)


'Expect' has two different meanings in this context.
1. The empirical behavior surprised me (because I did not bother to read 
the manual, which clearly says what the returns are).
2. The documented behavior, which I read, surprises me, because I would 
have designed things differently, perhaps because I have used other 
languages designed differently.


I am not sure which you meant.



--
http://mail.python.org/mailman/listinfo/python-list

Parser

[Andreas Löscher]

Hi,
I am looking for an easy to use parser. I am want to get an overview
over parsing and want to try to get some information out of a C-Header
file. Which parser would you recommend?

Best,
Andreas

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sometype.__new__ and C subclasses

[James Porter]

On 5/2/2010 1:43 PM, Robert Kern wrote:

Perhaps things would be clearer if you could post the C code that you've
written that fails. So far, you've only alluded at what you are doing
using Python-syntax examples.


I'm not sure how much this will help, but here you go. The actual C code 
probably doesn't matter except for where I set tp_flags, tp_new, and 
register the type, but I included it for completeness. The full C source 
is available here if you need it, but be warned that other strangeness 
abounds in the code: 
.


Obviously, this is kind of a bizarre case, so I'm not entirely sure what 
the best route is here.


Thanks,
Jim

static PyObject*
iMeshArrObj_new(PyTypeObject *cls,PyObject *args,PyObject *kw)
{
static char *kwlist[] = {"object","instance",0};

PyObject *obj;
iMesh_Object *instance = NULL;
PyObject *arr = NULL;
iMeshArr_Object *self;

if(!PyArg_ParseTupleAndKeywords(args,kw,"O|O!",kwlist,&obj,
&iMesh_Type,&instance))
return NULL;

arr = PyArray_FROM_O(obj);
if(arr == NULL)
return NULL;

self = (iMeshArr_Object*)PyObject_CallMethod(arr,"view","O",cls);
Py_DECREF(arr);
if(self == NULL)
return NULL;

/* some boring stuff to set |instance| */

return self;
}

static void
iMeshArrObj_dealloc(iMeshArr_Object *self)
{
Py_XDECREF(self->instance);
self->array.ob_type->tp_free((PyObject*)self);
}

static PyObject*
iMeshArrObj_finalize(iMeshArr_Object *self,PyObject *args)
{
iMeshArr_Object *context;
if(PyArg_ParseTuple(args,"O!",&iMeshArr_Type,&context))
{
self->instance = context->instance;
Py_XINCREF(self->instance);
}
PyErr_Clear();
Py_RETURN_NONE;
}

static PyMethodDef iMeshArrObj_methods[] = {
{ "__array_finalize__", (PyCFunction)iMeshArrObj_finalize,
  METH_VARARGS, ""
},
{0}
};

static PyMemberDef iMeshArrObj_members[] = {
{"instance", T_OBJECT_EX, offsetof(iMeshArr_Object, instance),
 READONLY, "base iMesh instance"},
{0}
};

static PyTypeObject iMeshArr_Type = {
PyObject_HEAD_INIT(NULL)
/* ... */
(destructor)iMeshArrObj_dealloc,  /* tp_dealloc */
/* ... */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /* tp_flags */
"iMesh array objects",/* tp_doc */
/* ... */
iMeshArrObj_methods,  /* tp_methods */
iMeshArrObj_members,  /* tp_members */
/* ... */
iMeshArrObj_new,  /* tp_new */
};

PyMODINITFUNC initiMesh(void)
{
PyObject *m;
m = Py_InitModule("iMesh",module_methods);
import_array();

iMeshArr_Type.tp_base = &PyArray_Type;  
if(PyType_Ready(&iMeshArr_Type) < 0)
return;
Py_INCREF(&iMeshArr_Type);
PyModule_AddObject(m,"Array",(PyObject *)&iMeshArr_Type);
}

/* End C code */

And then in Python:

A = iMesh.Array(numpy.array([1,2,3,4,5]), instance=mesh)
numpy.zeros_like(A) # fails here

Inside NumPy, zeros_like looks like this (there's a bit more than this, 
but it's irrelevant to this problem):


def zeros_like(a):
if isinstance(a, ndarray):
res = ndarray.__new__(type(a), a.shape, a.dtype,
  order=a.flags.fnc)
res.fill(0)
return res

--
http://mail.python.org/mailman/listinfo/python-list

Re: condition and True or False

[David Robinow]

On Sun, May 2, 2010 at 1:14 PM, Paul McGuire  wrote:
> While sifting through some code looking for old "x and y or z" code
> that might better be coded using "y if x else z", I came across this
> puzzler:
>
>    x =  and True or False
>
> What is "and True or False" adding to this picture?  The boolean
> expression part is already evaluating to a boolean, so I don't
> understand why a code author would feel compelled to beat this one
> over the head with the additional "and True or False".
>
> I did a little code Googling and found a few other Python instances of
> this, but also many Lua instances.  I'm not that familiar with Lua, is
> this a practice that one who uses Lua frequently might carry over to
> Python, not realizing that the added "and True or False" is redundant?
>
> Other theories?
>
> -- Paul
> --
> http://mail.python.org/mailman/listinfo/python-list
>
True and False were added in Python 2.2.1  (PEP 285)
Perhaps this was a silly way to ensure that the user wouldn't try to
run it in earlier versions.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Parser

[Jason Scheirer]

On May 2, 12:54 pm, Andreas Löscher  wrote:
> Hi,
> I am looking for an easy to use parser. I am want to get an overview
> over parsing and want to try to get some information out of a C-Header
> file. Which parser would you recommend?
>
> Best,
> Andreas

Pyparsing: http://pyparsing.wikispaces.com/
I've abused the lexers built in to Pygments ( http://pygments.org/ ) a
few times to decent effect too.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sometype.__new__ and C subclasses

[Carl Banks]

On May 2, 10:48 am, James Porter  wrote:
> On 5/2/2010 4:34 AM, Carl Banks wrote:
>
> > Why don't you use mysubtype.__new__(mysubtype,...)?
>
> > If you wrote mysubtype in C, and defined a different tp_new than
> > ndarray, then this exception will trigger.  And it ought to; you don't
> > want to use ndarray's tp_new to create an object of your subclass, if
> > you've defined a different tp_new.
>
> Unfortunately, I can't do that, since that call is in NumPy itself and
> it's part of their "standard" way of making instances of subclasses of
> ndarray. Functions like numpy.zeros_like use ndarray.__new__(subtype,
> ...) to create new arrays based on the shape of other arrays.
>
> The Python version of the subclass is shown here:
> ,
> and I'm trying to write something pretty similar in C. I'm trying to
> stay in C since everything else is in C, so it's easier to stay in C
> then to jump back and forth all the time.
>
> Maybe the real answer to this question is "NumPy is doing it wrong" and
> I should be on their list; still, it seems strange that the behavior is
> different between Python and C.

I would say numpy is wrong here, so I suggest filing a bug report.

In fact I can't think of any benefit to EVER calling X.__new__(Y)
where X is not Y.  Maybe old-style classes?  Someone who wants to
ensure they're getting an instance of a certain type can check
issubclass(Y,X) then call Y.__new__(Y).

Unfortunately, you just can't get rid of the test in tp_new_wrapper.


Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sometype.__new__ and C subclasses

[Robert Kern]

On 2010-05-02 15:28 , Carl Banks wrote:

On May 2, 10:48 am, James Porter  wrote:

On 5/2/2010 4:34 AM, Carl Banks wrote:


Why don't you use mysubtype.__new__(mysubtype,...)?



If you wrote mysubtype in C, and defined a different tp_new than
ndarray, then this exception will trigger.  And it ought to; you don't
want to use ndarray's tp_new to create an object of your subclass, if
you've defined a different tp_new.


Unfortunately, I can't do that, since that call is in NumPy itself and
it's part of their "standard" way of making instances of subclasses of
ndarray. Functions like numpy.zeros_like use ndarray.__new__(subtype,
...) to create new arrays based on the shape of other arrays.

The Python version of the subclass is shown here:
,
and I'm trying to write something pretty similar in C. I'm trying to
stay in C since everything else is in C, so it's easier to stay in C
then to jump back and forth all the time.

Maybe the real answer to this question is "NumPy is doing it wrong" and
I should be on their list; still, it seems strange that the behavior is
different between Python and C.


I would say numpy is wrong here, so I suggest filing a bug report.

In fact I can't think of any benefit to EVER calling X.__new__(Y)
where X is not Y.  Maybe old-style classes?  Someone who wants to
ensure they're getting an instance of a certain type can check
issubclass(Y,X) then call Y.__new__(Y).


Well, the Y.__new__(Y) may call X.__new__(Y) (and we certainly do this 
successfully in other Python subclasses of ndarray; this also appears in the 
Python regression tests). I'm not sure why this would be permitted there and not 
in a regular function (numpy.zeros_like() seems to be the function that does 
this and fails for the OP). The reason we do it there instead of calling the 
subclass's constructor is because the subclass's constructor may have different 
arguments.


I'm happy to concede that this might be a bug in numpy, but I don't understand 
why this is allowed for Python subclasses but not C subtypes.


--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list

Re: sometype.__new__ and C subclasses

[Robert Kern]

On 2010-05-02 15:03 , James Porter wrote:


And then in Python:

A = iMesh.Array(numpy.array([1,2,3,4,5]), instance=mesh)
numpy.zeros_like(A) # fails here

Inside NumPy, zeros_like looks like this (there's a bit more than this,
but it's irrelevant to this problem):

def zeros_like(a):
if isinstance(a, ndarray):
res = ndarray.__new__(type(a), a.shape, a.dtype,
order=a.flags.fnc)
res.fill(0)
return res


Well, I think we can change zeros_like() and the rest to work around this issue. 
Can you bring it up on the numpy mailing list?


def zeros_like(a):
if isinstance(a, ndarray):
res = numpy.empty(a.shape, a.dtype, order=a.flags.fnc)
res.fill(0)
res = res.view(type(a))
return res
...

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list

Re: PyObject_SetAttrString - doesn't set instance attribute

[Carl Banks]

On May 2, 3:26 am, Jason  wrote:
> On May 2, 5:52 pm, Carl Banks  wrote:
>
> > Not sure what you're doing here.  It looks like you are being passed
> > an object of a given type, then you get the type object, call it to
> > create another object of that type, and assign it to object->instance.
>
> Sorry, I should have noted that the "NautilusPythonObject" type in the
> code is a struct defined as:
>
> struct _NautilusPythonObject {
>   GObject parent_slot;
>   PyObject *instance;
>
> };
> > You are assigning the attirbute the the object that the C code refers
> > to as "object->instance", but it seems that in the Python snippet you
> > are calling getattr on the object that the C code refers to as
> > "object".
>
> object->instance is the PyObject, and I gathered that it was the
> correct thing to assign to from the fact that the address is identical
> as seen from C and Python.

We'd have to see more code, I'd think.  What you posted needs more
context.  For instance, what type, exactly, is class->type?  This
would help us understand better.

I don't understand how object->instance and self could be the same
object.  If they have the same address it's possible (and, I'm
inclined to think, likely) that you're creating an object, it's
getting destroyed, then you are creating another one.

Here's what's really odd.  You are calling getattr(self,
"super_happy_magic") inside __init__, which is the class's
initializer.  How could you have had occasion to call
PyObject_SetAttrString on that same object at that point?  The only
possible way it could have happened is if
nautilus_python_object_instance_init is invoked by
MenuProviderTest.__new__ somehow, but that doesn't make sense either.
You run PyObject_CallObject(class->type,NULL) to create object-
>instance, but calling a type also calls the type's __init__ method.
So how is it that later the __init__ method is being called again on
the same object?  Unless you're doing something very weird, it could
only mean it's a different object.

I doubt it'll fix all your problems, but one thing to try is to
replace "PyObject_CallObject(class->type, NULL);" with "class->type-
>tp_new(class->type);".

But you probably have to go back to the drawing board and rethink the
whole thing.  What you've posted is quite unusual.


You should choose more descriptive variable names, too.


Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sometype.__new__ and C subclasses

[Carl Banks]

On May 2, 1:51 pm, Robert Kern  wrote:
> On 2010-05-02 15:28 , Carl Banks wrote:
>
>
>
> > On May 2, 10:48 am, James Porter  wrote:
> >> On 5/2/2010 4:34 AM, Carl Banks wrote:
>
> >>> Why don't you use mysubtype.__new__(mysubtype,...)?
>
> >>> If you wrote mysubtype in C, and defined a different tp_new than
> >>> ndarray, then this exception will trigger.  And it ought to; you don't
> >>> want to use ndarray's tp_new to create an object of your subclass, if
> >>> you've defined a different tp_new.
>
> >> Unfortunately, I can't do that, since that call is in NumPy itself and
> >> it's part of their "standard" way of making instances of subclasses of
> >> ndarray. Functions like numpy.zeros_like use ndarray.__new__(subtype,
> >> ...) to create new arrays based on the shape of other arrays.
>
> >> The Python version of the subclass is shown here:
> >> ,
> >> and I'm trying to write something pretty similar in C. I'm trying to
> >> stay in C since everything else is in C, so it's easier to stay in C
> >> then to jump back and forth all the time.
>
> >> Maybe the real answer to this question is "NumPy is doing it wrong" and
> >> I should be on their list; still, it seems strange that the behavior is
> >> different between Python and C.
>
> > I would say numpy is wrong here, so I suggest filing a bug report.
>
> > In fact I can't think of any benefit to EVER calling X.__new__(Y)
> > where X is not Y.  Maybe old-style classes?  Someone who wants to
> > ensure they're getting an instance of a certain type can check
> > issubclass(Y,X) then call Y.__new__(Y).
>
> Well, the Y.__new__(Y) may call X.__new__(Y) (and we certainly do this
> successfully in other Python subclasses of ndarray; this also appears in the
> Python regression tests). I'm not sure why this would be permitted there and 
> not
> in a regular function (numpy.zeros_like() seems to be the function that does
> this and fails for the OP). The reason we do it there instead of calling the
> subclass's constructor is because the subclass's constructor may have 
> different
> arguments.
>
> I'm happy to concede that this might be a bug in numpy, but I don't understand
> why this is allowed for Python subclasses but not C subtypes.

Because Python subclasses (i.e., "heap types") all invoke
tp_new_wrapper, which is guaranteed to call the tp_new of the most
derived base.

C subtypes can, and often have to, replace tp_new with their own
version.  Calling a base type's tp_new when you've defined your own
tp_new at the C level is dangerous.


As for the issue with a subclass's arguments being different, I'm
shocked that anyone at numpy could possibly think bypassing the
subtype's constructor is good idea.


Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: condition and True or False

[Carl Banks]

On May 2, 10:14 am, Paul McGuire  wrote:
> While sifting through some code looking for old "x and y or z" code
> that might better be coded using "y if x else z", I came across this
> puzzler:
>
>     x =  and True or False
>
> What is "and True or False" adding to this picture?  The boolean
> expression part is already evaluating to a boolean, so I don't
> understand why a code author would feel compelled to beat this one
> over the head with the additional "and True or False".
>
> I did a little code Googling and found a few other Python instances of
> this, but also many Lua instances.  I'm not that familiar with Lua, is
> this a practice that one who uses Lua frequently might carry over to
> Python, not realizing that the added "and True or False" is redundant?
>
> Other theories?

The person who wrote it was a programmer who fancied himself to be
more clever than he really was.

Convenient though they may be, people always abuse (and often misuse)
these boolean shortcuts, a big reason why I don't like them.  You
never see crap like this in Java.


Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Parser

[Martin v. Loewis]

Andreas Löscher wrote:
> Hi,
> I am looking for an easy to use parser. I am want to get an overview
> over parsing and want to try to get some information out of a C-Header
> file.

To get information from a header file, try Tools/scripts/h2py.py

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sometype.__new__ and C subclasses

[James Porter]

On 5/2/2010 3:58 PM, Robert Kern wrote:

Well, I think we can change zeros_like() and the rest to work around
this issue. Can you bring it up on the numpy mailing list?

def zeros_like(a):
if isinstance(a, ndarray):
res = numpy.empty(a.shape, a.dtype, order=a.flags.fnc)
res.fill(0)
res = res.view(type(a))
return res
...


I'm having difficulty posting to the NumPy list (both via gmane and 
email) so I'm just going to put this here so it doesn't get lost. 
zeros_like probably needs to call __array_finalize__ for this to work 
properly (it'll cause a segfault for me otherwise):


 def zeros_like(a):
 if isinstance(a, ndarray):
 res = numpy.zeros(a.shape, a.dtype, order=a.flags.fnc)
 res = res.view(type(a))
 res.__array_finalize__(a)
 return res
 ...

- Jim

--
http://mail.python.org/mailman/listinfo/python-list

Re: condition and True or False

[Steven D'Aprano]

On Sun, 02 May 2010 10:14:44 -0700, Paul McGuire wrote:

> While sifting through some code looking for old "x and y or z" code that
> might better be coded using "y if x else z", I came across this puzzler:
> 
> x =  and True or False
> 
> What is "and True or False" adding to this picture?  The boolean
> expression part is already evaluating to a boolean, so I don't
> understand why a code author would feel compelled to beat this one over
> the head with the additional "and True or False".

If  is already an actual bool, then I can only 
imagine that the author is simply struggling with the concept, in the 
same way that some people write:

if  == True:
...


If it is any arbitrary object, then "x and True or False" is just an 
obfuscated way of writing "bool(x)". Perhaps their code predates the 
introduction of bools, and they have defined global constants True and 
False but not bool. Then they removed the True and False bindings as no 
longer necessary, but neglected to replace the obfuscated conversion.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

ANN: expy 0.6.6 released!

[Yingjie Lan]

EXPY is an express way to extend Python!

EXPY provides a way to extend python in an elegant way. For more information 
and a tutorial, see: http://expy.sourceforge.net/

What's new:

1. Special methods can now take @throws decorators.
2. Added convenience macros _NEW and _CheckExact for extension 
types.
3. Give warnings of missing Py_INCREF on all methods/functions returning an 
object. 
4. And the responsibility of Py_INCREF is left for the developer.
5. Documentation update.

Cheers,

Yingjie


  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: List size versus list allocation

[Steven D'Aprano]

On Sun, 02 May 2010 18:07:31 +0200, Christian Heimes wrote:

>> I'm interested in gathering some statistics on this, and to do so I
>> need a way of measuring the list's logical size versus its actual size.
>> The first is easy: it's just len(list). Is there some way of getting
>> the allocated size of the list?
> 
> With Python 2.6 and newer you can use sys.getsizeof() to get the actual
> size of the list object:


Thanks Christian and Peter for your answers. That's very helpful.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: condition and True or False

[Patrick Maupin]

On May 2, 12:14 pm, Paul McGuire  wrote:
> While sifting through some code looking for old "x and y or z" code
> that might better be coded using "y if x else z", I came across this
> puzzler:
>
>     x =  and True or False
>
> What is "and True or False" adding to this picture?  The boolean
> expression part is already evaluating to a boolean, so I don't
> understand why a code author would feel compelled to beat this one
> over the head with the additional "and True or False".
>
> I did a little code Googling and found a few other Python instances of
> this, but also many Lua instances.  I'm not that familiar with Lua, is
> this a practice that one who uses Lua frequently might carry over to
> Python, not realizing that the added "and True or False" is redundant?
>
> Other theories?
>
> -- Paul

I think it's idiomatic -- that it was written by someone who was deep
in thought about actually getting something accomplished, and not
thinking at the level of the details.

As you're actively *looking* for "x and y or z" I'm sure you'll agree
that we've probably all written lots of stuff like:

x =  and 'some_prefix' or ''
x =  and 42 or 0
x =  and ['Hi, mom!'] or []

When you're in this mode of expression, the only thing that would
really trip you up and make it "wrong" is if 'some_prefix' or 42 or
['Hi, mom!'] evaluated to False, and then you get the '', 0, or [] you
didn't really want.  So I know that, to the extent I was thinking
deeply about the low level of what I was writing, my mental energy
would be going towards making sure that the second sub-expression
evaluated to true.

So, if you're mentally operating in this mode, and you want True or
False, and you forget, don't think about, or are not cognizant of the
fact that bool() is available, it's pretty obvious what the result is
going to be.  I've probably done it myself a few times, although I
would probably have to be *really* lost in thought in order to do it
when the underlying sub-expression was boolean to start with.

Regards,
Pat
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sometype.__new__ and C subclasses

[Gregory Ewing]

James Porter wrote:
Functions like numpy.zeros_like use ndarray.__new__(subtype, 
...) to create new arrays based on the shape of other arrays.


Maybe the real answer to this question is "NumPy is doing it wrong"


Yes, I think NumPy is doing it wrong, even for subclasses
written in Python. If the subtype has overridden ndarray's
__new__ method, the way NumPy is doing it will skip the
subclass's version of the method.

--
Greg
--
http://mail.python.org/mailman/listinfo/python-list

Re: condition and True or False

[John Machin]

On May 3, 9:14 am, Steven D'Aprano  wrote:

> If it is any arbitrary object, then "x and True or False" is just an
> obfuscated way of writing "bool(x)". Perhaps their code predates the
> introduction of bools, and they have defined global constants True and
> False but not bool. Then they removed the True and False bindings as no
> longer necessary, but neglected to replace the obfuscated conversion.

Or perhaps they are maintaining code that must run on any 2.X. True
and False would be set up conditional on Python version. Writing
"expression and True or False" avoids a function call.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: ANN: expy 0.6.6 released!

[Yingjie Lan]

> Subject: ANN: expy 0.6.6 released!
> To: "python list" 
> Cc: "CAPI Python" 
> Date: Monday, May 3, 2010, 3:24 AM
> EXPY is an express way to extend Python!
> 
> EXPY provides a way to extend python in an elegant way. For
> more information and a tutorial, see: http://expy.sourceforge.net/
> 

I'm using expy in a serious project to wrap an old project written in C and 
deliver it up via www with django. That is why expy is getting improved quickly 
these days. So far, both the project and expy are making good progress hand in 
hand.

Cheers,

Yingjie


  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Fast Efficient way to transfer an object to another list

[Bryan]

Steven D'Aprano wrote:
> The simplest way to speed the above code up is not to start from the
> beginning each time. That requires two very small changes. And since
> deletions from the front of the list are slow, MRAB's suggestion is also
> a good idea.

Those two speed-ups provide worst-case linear run-time, but as MRAB
astutely noted, his optimization assumes that order is unimportant.
That's a bad assumption for a general extract-from-list function.
Where order does not matter, I'd suspect 'list' was a poor choice of
data type. Here's a general, order-preserving, linear-time version:


def extract(lst, predicate):
""" Return items of lst satisfying predicate, deleting them from
lst.
"""
result = []
j = 0
for i in range(len(lst)):
if predicate(lst[i]):
result.append(lst[i])
else:
lst[j] = lst[i]
j += 1
del lst[j:]
return result


# some testing:
for nitems in range(10):
for pred in [lambda x: True,
lambda x: False,
lambda x: x % 2 == 0,
lambda x: x % 2 == 1,
lambda x: x < nitems // 2,
lambda x: x >= nitems // 2]:
original = list(range(nitems))
source = original[:]
extracted = extract(source, pred)
assert len(source) + len(extracted) == nitems
assert sorted(source) == source
for n in source:
assert not pred(n)
assert n in original
assert sorted(extracted) == extracted
for n in extracted:
assert pred(n)
assert n in original



--
--Bryan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python dot-equals (syntax proposal)

[Alf P. Steinbach]

* Terry Reedy:

* Alf P. Steinbach:

* Aahz:



and sometimes
they rebind the original target to the same object.


At the Python level that seems to be an undetectable null-operation.


If you try t=(1,2,3); t[1]+=3, if very much matters that a rebind occurs.


Testing:


>>> t = ([], [], [])
>>> t
([], [], [])
>>> t[0] += ["blah"]
Traceback (most recent call last):
  File "", line 1, in 
TypeError: 'tuple' object does not support item assignment
>>> t
(['blah'], [], [])
>>> _


Yep, it matters.

Is this change-but-raise-exception a bug?

I seem to have a knack for running into bugs. :-)



Granted one could see something going on in a machine code or byte code
debugger. But making that distinction (doing nothing versus
self-assignment) at the Python level seems, to me, to be meaningless.


Please do not confuse things. Augmented *assignment* must be understood
as assignment. Failure to do so leads (and has lead) newbies into
confusion, and puzzled posts on this list.


OK.

But I think it would be less confusing, less breaking of expectations, if, for 
the example above, += reduced to the functionality of extend(), with no x.



Cheers, & thanks,

- Alf
--
http://mail.python.org/mailman/listinfo/python-list

Re: Parser

[dmtr]

On May 2, 12:54 pm, Andreas Löscher  wrote:
> Hi,
> I am looking for an easy to use parser. I am want to get an overview
> over parsing and want to try to get some information out of a C-Header
> file. Which parser would you recommend?

ANTLR
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Parser

[Chris Rebert]

On Sun, May 2, 2010 at 10:10 PM, dmtr  wrote:
> On May 2, 12:54 pm, Andreas Löscher  chemnitz.de> wrote:
>> Hi,
>> I am looking for an easy to use parser. I am want to get an overview
>> over parsing and want to try to get some information out of a C-Header
>> file. Which parser would you recommend?
>
> ANTLR

I don't know if it's that easy to get started with though. The
companion for-pay book is *most excellent*, but it seems to have been
written to the detriment of the normal online docs.

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Parser

[dmtr]

>
> > ANTLR
>
> I don't know if it's that easy to get started with though. The
> companion for-pay book is *most excellent*, but it seems to have been
> written to the detriment of the normal online docs.
>
> Cheers,
> Chris
> --http://blog.rebertia.com


IMO ANTLR is much easier to use compared to any other tool simply
because it has excellent GUI (the quality of which is amazing).
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Parser

[Stefan Behnel]

dmtr, 03.05.2010 07:39:

ANTLR


I don't know if it's that easy to get started with though. The
companion for-pay book is *most excellent*, but it seems to have been
written to the detriment of the normal online docs.


IMO ANTLR is much easier to use compared to any other tool simply
because it has excellent GUI (the quality of which is amazing).


See? Explanations help more than bold statements.

Stefan

--
http://mail.python.org/mailman/listinfo/python-list

Problems with 'scipy.sparse.linalg.dsolve.umfpack'

[pp]

Hello all,

I have this equation to solve where variable
LHS is a sparse matrix and RHS is an array .
c = linsolve.spsolve(LHS, RHS)


i get this error
warn( 'scipy.sparse.linalg.dsolve.umfpack will be removed,'\
  ' install scikits.umfpack instead'
what is the reason for this and do I do next

Please help
I am very new to scipy.

Thanks a lot
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Problems with 'scipy.sparse.linalg.dsolve.umfpack'

[Chris Rebert]

On Sun, May 2, 2010 at 11:24 PM, pp  wrote:
> Hello all,
>
> I have this equation to solve where variable
> LHS is a sparse matrix and RHS is an array .
> c = linsolve.spsolve(LHS, RHS)
>
>
> i get this error
> warn( 'scipy.sparse.linalg.dsolve.umfpack will be removed,'\
>                  ' install scikits.umfpack instead'
> what is the reason for this and do I do next
>
> Please help
> I am very new to scipy.

This is the general Python mailinglist. You'd be more likely to get an
answer on the SciPy mailinglist:
http://mail.scipy.org/mailman/listinfo/scipy-user

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: recommended way to insert data into a one to many relationship using python

[Wolfgang Meiners]

Peter Otten schrieb:
> Peter Otten wrote:
> 
> If you create indices for floors (and rooms)
> 
> cur.execute("""create unique index room_index on rooms (fid, number);""")
> cur.execute("""create unique index floor_index on floors (floor);""")
> 
> the addition of missing rows can be simplified to
> 
> missing = c2.execute("""select distinct floor from new_employees;""")
> cur.executemany("insert or ignore into floors (floor) values (?)", missing)
> 
> etc.
> 
> Peter

Hi Peter,
thank you for your response.
What i have got from it, is to have a (temporary) table to do the work
inside sql and not from python. I thought of a second method to do it
inside sql by an trigger

sql = """create trigger insert_new_employee instead of insert on emplist
begin
  # insert floor if not exists floor
  # insert (room, floor) if not exists (room, floor)
  # insert (person, rid)
end"""

but i would have to learn how to write triggers. Your idea gives me a
more direct solution.

Wolfgang
-- 
http://mail.python.org/mailman/listinfo/python-list