bug in copy.deepcopy or in getattr or in my understanding?

[Emin]

Dear experts,

I got some unexpected behavior in getattr and copy.deepcopy (see
transcript below). I'm not sure if this is actually a bug in
copy.deepcopy or if I'm doing something too magical with getattr.
Comments would be appreciated.

Thanks,
-Emin

#  Transcript follows ##


Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.

>>> # THE FOLLOWING BREAKS
>>> class a:
... def foo(self):
... print 'hi'
...
>>> class b(a):
... def __init__(self):
... self.y = getattr(self,'foo')
...
>>> c = b()
>>> import copy
>>> copy.deepcopy(c)
Traceback (most recent call last):
  File "", line 1, in 
  File "c:\Python25\lib\copy.py", line 162, in deepcopy
y = copier(x, memo)
  File "c:\Python25\lib\copy.py", line 291, in _deepcopy_inst
state = deepcopy(state, memo)
  File "c:\Python25\lib\copy.py", line 162, in deepcopy
y = copier(x, memo)
  File "c:\Python25\lib\copy.py", line 254, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "c:\Python25\lib\copy.py", line 189, in deepcopy
y = _reconstruct(x, rv, 1, memo)
  File "c:\Python25\lib\copy.py", line 322, in _reconstruct
y = callable(*args)
  File "c:\Python25\lib\copy_reg.py", line 92, in __newobj__
return cls.__new__(cls, *args)
TypeError: instancemethod expected at least 2 arguments, got 0
>>> # THE FOLLOWING WORKS
>>> class b(a):
... def __init__(self):
... self.x = self.__class__.__bases__[0].__dict__['foo']
... 
>>> c=b()
>>> copy.deepcopy(c)
<__main__.b instance at 0x00EADE18>

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: bug in copy.deepcopy or in getattr or in my understanding?

[Emin]

Dear Gabriel,

Thank you for your reply. As you guessed, I want to be able to select
the method at runtime as in your final example, but when I tried your
suggestion I got the same error (see below). I think the problem is
that getattr is donig something different than in my example where I
explicitly get it from the dict (see the very end of the transcript
below):


--- Transcript Follows
--

Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> # The following shows that using getattr to grab a method is
>>> # incompatible with copy.deepcopy
>>> class a:
... def foo(self): pass
...
>>> class b(a):
... def __init__(self):
... self.x = getattr(a,'foo')
...
>>> import copy
>>> c=b()
>>> copy.deepcopy(c)
Traceback (most recent call last):
  File "", line 1, in 
  File "c:\Python25\lib\copy.py", line 162, in deepcopy
y = copier(x, memo)
  File "c:\Python25\lib\copy.py", line 291, in _deepcopy_inst
state = deepcopy(state, memo)
  File "c:\Python25\lib\copy.py", line 162, in deepcopy
y = copier(x, memo)
  File "c:\Python25\lib\copy.py", line 254, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "c:\Python25\lib\copy.py", line 189, in deepcopy
y = _reconstruct(x, rv, 1, memo)
  File "c:\Python25\lib\copy.py", line 322, in _reconstruct
y = callable(*args)
  File "c:\Python25\lib\copy_reg.py", line 92, in __newobj__
return cls.__new__(cls, *args)
TypeError: instancemethod expected at least 2 arguments, got 0
>>>
>>> # The following shows that getattr is doing something different
>>> # than looking in the __dict__ of base classes
>>> b.__bases__[0].__dict__['foo']

>>> getattr(a,'foo')

>>>


On Jan 4, 10:08 pm, Gabriel Genellina <[EMAIL PROTECTED]> wrote:
> At Thursday 4/1/2007 17:26, Emin wrote:
>
> >I got some unexpected behavior in getattr and copy.deepcopy (see
> >transcript below). I'm not sure if this is actually a bug in
> >copy.deepcopy or if I'm doing something too magical with getattr.
> >Comments would be appreciated.Both examples are different. #1 stores a 
> >*bound* method into an
> instance attribute. Bound methods carry a reference to "self", this
> creates a cyclic reference that may cause problems to the garbage
> collector (and confuses deepcopy, apparently).
> #2 uses and *unbound* method and it's the usual way.
>
> > >>> class a:
> >... def foo(self):
> >... print 'hi'
> >...
> > >>> class b(a): #1
> >... def __init__(self):
> >... self.y = getattr(self,'foo')
>
> > >>> class b(a): #2
> >... def __init__(self):
> >... self.x = self.__class__.__bases__[0].__dict__['foo']For #2 
> >you can simply say:
>
> class b(a):
>  x = a.foo
>
> If you have to choose at runtime (a simplified version of your own code):
>
> class b(a):
>  def __init__(self):
>  name = select_method_to_use(..., default="foo")
>  self.x = getattr(a, name)
>
> You *know* your bases because you wrote them in the class statement
> (or use super() instead of playing with __bases__); and getattr works
> fine here so you don't need to mess with the __dict__ details.
>
> (Note that #1 was *almost* right, you had to replace self by the base class)
>
> --
> Gabriel Genellina
> Softlab SRL
>
> __
> Preguntá. Respondé. Descubrí.
> Todo lo que querías saber, y lo que ni imaginabas,
> está en Yahoo! Respuestas (Beta).
> ¡Probalo ya!http://www.yahoo.com.ar/respuestas

-- 
http://mail.python.org/mailman/listinfo/python-list

what is the idiom for copy lots of params into self?

[Emin]

Dear Experts,

When writing large classes, I sometimes find myself needing to copy a
lot of parameters from the argument of __init__ into self. Instead of
having twenty lines that all basically say something like self.x = x, I
often use __dict__ via something like:

class example:
 def __init__(self,a,b,c,d,e,f,g,h,i,j,k,l,m,n):
 for name in
['a','b','c','d','e','f','g','h','i','j','k','l','m','n']:
 self.__dict__[name] = locals()[name]

This saves a lot of code and makes it easier to see what is going on,
but it seems like there should be a better idiom for this task. Any
suggestions?

Thanks,
-Emin

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: what is the idiom for copy lots of params into self?

[Emin]

Dear Luis and everyone else who responded,

Thanks for your suggestions. One issue with using *args or **kw is that
I might no want to copy all the arguments to __init__ into self.

What made me ask the question in my original post was not so much that
I had to loop over the names I wanted to save, but whether it's okay to
mess with self.__dict__ or if there is another way I should be
assigning to self.

Thanks,
-Emin

On Jan 10, 9:05 pm, "Luis M. González" <[EMAIL PROTECTED]> wrote:
> Emin wrote:
> > Dear Experts,
>
> > When writing large classes, I sometimes find myself needing to copy a
> > lot of parameters from the argument of __init__ into self. Instead of
> > having twenty lines that all basically say something like self.x = x, I
> > often use __dict__ via something like:
>
> > class example:
> >  def __init__(self,a,b,c,d,e,f,g,h,i,j,k,l,m,n):
> >  for name in
> > ['a','b','c','d','e','f','g','h','i','j','k','l','m','n']:
> >  self.__dict__[name] = locals()[name]
>
> > This saves a lot of code and makes it easier to see what is going on,
> > but it seems like there should be a better idiom for this task. Any
> > suggestions?
>
> > Thanks,
> > -EminHow about using variable length argumens?
>
> class example:
> def __init__(self, *args):
> for i in args:
> self.__dict__[i] = i
> 
> x = example('uno','dos','tres')
> 
> Luis

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: what is the idiom for copy lots of params into self?

[Emin]

Thanks, that looks like what I wanted.

On Jan 11, 8:36 am, "Fredrik Lundh" <[EMAIL PROTECTED]> wrote:
> "Emin" <[EMAIL PROTECTED]> wrote:
> > What made me ask the question in my original post was not so much that
> > I had to loop over the names I wanted to save, but whether it's okay to
> > mess with self.__dict__ or if there is another way I should be
> > assigning to self.   http://effbot.org/pyref/setattr
> 
> 

-- 
http://mail.python.org/mailman/listinfo/python-list

How much slower is dict indexing vs. list indexing?

[Emin]

Dear Experts,

How much slower is dict indexing vs. list indexing (or indexing into a
numpy array)? I realize that looking up a value in a dict should be
constant time, but does anyone have a sense of what the overhead will
be in doing a dict lookup vs. indexing into a list? Some ad hoc tests
I've done indicate that the overhead is less than 15% (i.e., dict
lookups seem to take no more than 15% longer than indexing into a list
and there doesn't seem to be much difference in indexing into a list
vs. a numpy array).

The reason I ask is because I'm wondering how hard I should work to
compute and store an index into a list to avoid repeated hash table
lookups. From my tests, it looks like the answer is basically "don't
bother". Does anyone have information, thoughts, or comments on this?

Thanks,
-Emin

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How much slower is dict indexing vs. list indexing?

[Emin]

On Jan 11, 5:53 pm, Steve Holden <[EMAIL PROTECTED]> wrote:
>
> What technique were you thinking of to look up the cached index values
> in Python, just as a matter of curiosity? Storing them in a dict? It
> would be hard to think of a faster way ... ;-)

I didn't have anything fancy in mind. I was just wondering whether it
makes sense to replace a block of code like

data = {'a' : 1, 'b' :2, 'c' : 3}
for i in someLargeList:
   for name in ['a','b','c']:
  result.append(data[name])

with somthing like

data = {'a' : 1, 'b' :2, 'c' : 3}
dataValues = [data[k] for k in ['a','b','c']
for i in someLargeList:
   for index in [0,1,2]:
  result.append(dataValues[index])

It sounds like it's probably not worth the effort in general, but might
be for extremely ultra-critical parts of code.

Thanks

-- 
http://mail.python.org/mailman/listinfo/python-list

question about module resolution

[Emin]

Dear Experts,

I often find myself wanting to have a child module get some parameters
defined in a parent module. For example, imagine  I have the following
directory structure and want something in baz.py to look at a value in
config.py. I end up putting in things like import sys;
sys.path.append('../..'). Is there a better way?

foo/
 __init__.py
 config.py
 bar/
  __init__.py
  baz.py

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: question about module resolution

[Emin]

I put the lines you suggested in baz.py, but got an error:

Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import baz
Traceback (most recent call last):
  File "", line 1, in 
  File "baz.py", line 3, in 
from .. import config
ValueError: Attempted relative import in non-package

On Jan 17, 11:20 am, Peter Otten <[EMAIL PROTECTED]> wrote:
> Emin wrote:
> > I often find myself wanting to have a child module get some parameters
> > defined in a parent module. For example, imagine  I have the following
> > directory structure and want something in baz.py to look at a value in
> > config.py. I end up putting in things like import sys;
> > sys.path.append('../..'). Is there a better way?
>
> > foo/
> >  __init__.py
> >  config.py
> >  bar/
> >   __init__.py
> >   baz.pyfrom __future__ import absolute_import
> from .. import config
> 
> Peter

-- 
http://mail.python.org/mailman/listinfo/python-list

emacs shell hangs on W32 with python

[emin . shopper]

Emacs seems to freeze when doing certain shell commands on Microsoft
Windows. The following is a simple example with Xemacs:

--

[Xemacs version 21.4.19; January 2006]

Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

h:\>c:\python25\python.exe -i
c:\python25\python.exe -i
Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import Tkinter
>>>
>>>
>>> t = Tkinter.Tk()
>>> 2
2
2

---
At this point, python no longer accepts input. This example works fine
when starting python through dos instead of emacs. Note, I am not using
python-mode.el so that is not the problem. I also reproduced the
problem with GNU Emacs (version "GNU Emacs 21.3.1
(i386-mingw-nt5.1.2600)
 of 2004-03-10 on NYAUMO")


Is there any way to run python through emacs or xemacs without having
it hang or is shell support broken?

Thanks in advance,
-Emin

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: emacs shell hangs on W32 with python

[emin . shopper]

Hmm, it doesn't even work if I run the cygwin version of GNU Emacs and
have it start the Windows version of python through python-mode or if I
start a shell in emacs and then start the Windows version of python
through the shell. Interestingly, however, things DO seem to work if I
use cygwin emacs with cygwin python (version 2.4.1).

I wonder if the windows version of Tkinter does something weird such as
implicitly calling Tk.mainloop()...

Any help would be much appreciated.

Brian Elmegaard wrote:
> [EMAIL PROTECTED] writes:
>
> > Is there any way to run python through emacs or xemacs without having
> > it hang or is shell support broken?
>
> Doing it from eshell gives the same problem :-(
> --
> Brian (remove the sport for mail)
> http://www.et.web.mek.dtu.dk/Staff/be/be.html
> Rugbyklubben Speed Scandinavian Open 7s Rugby http://www.rkspeed.dk

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: emacs shell hangs on W32 with python

[emin . shopper]

The same problems occur with the latest emacs binary from CVS
(emacs-version)

"GNU Emacs 23.0.0.1 (i386-mingw-nt5.1.2600) of 2006-10-16 on DTOP"

Lennart Borgman wrote:
> Would it not be a good idea to try the CVS version of Emacs for this now
> since the pretest will soon begin? Go here for precompiled versions:
>
> http://www.emacswiki.org/cgi-bin/wiki/CategoryWThirtyTwo
>
>
>
>
> [EMAIL PROTECTED] wrote:
> > Hmm, it doesn't even work if I run the cygwin version of GNU Emacs and
> > have it start the Windows version of python through python-mode or if I
> > start a shell in emacs and then start the Windows version of python
> > through the shell. Interestingly, however, things DO seem to work if I
> > use cygwin emacs with cygwin python (version 2.4.1).
> >
> > I wonder if the windows version of Tkinter does something weird such as
> > implicitly calling Tk.mainloop()...
> >
> > Any help would be much appreciated.
> >
> > Brian Elmegaard wrote:
> >
> >> [EMAIL PROTECTED] writes:
> >>
> >>
> >>> Is there any way to run python through emacs or xemacs without having
> >>> it hang or is shell support broken?
> >>>
> >> Doing it from eshell gives the same problem :-(
> >> --
> >> Brian (remove the sport for mail)
> >> http://www.et.web.mek.dtu.dk/Staff/be/be.html
> >> Rugbyklubben Speed Scandinavian Open 7s Rugby http://www.rkspeed.dk
> >>

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: emacs shell hangs on W32 with python

[emin . shopper]

Lennart Borgman wrote:
> Could you then please post a bug report? Just choose "Help - Send bug
> report" from the menus.

I sent a bug-report to the emacs list as requested and got an email
reply saying the message is being held for a moderator to look at.
Since the bug seems to be in the interaction of emacs with python, I'm
also cross-posting this to comp.lang.python in case anyone there can
help. (Note this bug shows up in all the versions of emacs that I have
tried including Xemacs and GNU Emacs 21).

The body of the bug-report is below:

I started emacs with -q did ESC-x shell and entered the following

--

Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

h:\projects\arcp_db\trunk\src\gui>c:\python25\python.exe -i
c:\python25\python.exe -i
Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import Tkinter
>>> t = Tkinter.Tk()
>>> 2

--

Note that after the call to Tkinter.Tk() my python window freezes and
no longer works. Killing the Tk window does not help. I realize that
this may seem like a bug in python and not emacs except that if I do
exactly the same thing through a dos shell I have no problems (i.e.,
python keeps accepting input). Consequently, it seems like emacs is
doing something funky when running the shell that interferes with
python in a way that running the shell without emacs would not do.


Sincerely,
-Emin Martinian


In GNU Emacs 23.0.0.1 (i386-mingw-nt5.1.2600)
 of 2006-10-16 on DTOP
X server distributor `Microsoft Corp.', version 5.1.2600
configured using `configure --with-gcc (4.1) --cflags  -O2 -g0
-march=i386 -mtune=i686 -pipe -IC:/gnuwin32/include_emacs
-IC:/gnuwin32/lib -IC:/gnuwin32/src --ldflags  -s '

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: ENU
  locale-coding-system: cp1252
  default-enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  encoded-kbd-mode: t
  tooltip-mode: t
  tool-bar-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  global-auto-composition-mode: t
  auto-composition-mode: t
  auto-compression-mode: t
  line-number-mode: t

Recent input:





Recent messages:
(c:\Program Files\emacs-23\ntemacs23\bin\emacs.exe -q)
Loading encoded-kb...done
For information about the GNU Project and its goals, type C-h C-p. [2
times]
Loading emacsbug...
Loading regexp-opt...done
Loading emacsbug...done

-- 
http://mail.python.org/mailman/listinfo/python-list

"initializer element is not constant"

[emin . martinian]

When trying to compile python extensions written in C using "python
setup.py build" on cygwin I get the following error:

foo.c: initializer element is not constant
foo.c: error: (near initialization for `FooType.ob_type')

I remember someone telling me a long time ago that this had something
to do with declspec and how dlls are imported on cygwin/python.  Can
someone give me a pointer to how to build python extensions on cygwin?

Thanks

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: I'm looking for a pythonic red-black tree...

[emin . shopper]

You could try and wrap the C/C++ code at
http://web.mit.edu/~emin/www/source_code/index.html and make a python
extension...

On Dec 14, 8:20 pm, "Just Another Victim of the Ambient Morality"
<[EMAIL PROTECTED]> wrote:
> I need a red-black tree in Python and I was wondering if there was one
> built in or if there's a good implementation out there.  Something that,
> lets face it, does whatever the C++ std::map<> allows you to do...
> Thank you...

-- 
http://mail.python.org/mailman/listinfo/python-list

module to implement Abstract Base Class

[emin . shopper]

I had a need recently to check if my subclasses properly implemented
the desired interface and wished that I could use something like an
abstract base class in python. After reading up on metaclass magic, I
wrote the following module. It is mainly useful as a light weight tool
to help programmers catch mistakes at definition time (e.g., forgetting
to implement a method required by the given interface). This is handy
when unit tests or running the actual program take a while.

Comments and suggestions are welcome.

Thanks,
-Emin


###  Abstract Base Class Module
Follows

"""
This module provides the AbstractBaseClass class and the Abstract
decorator
to allow you to define abstract base classes in python. See the
documentation
for AbstractBaseClass for instructions.
"""

class _AbstractMetaClass(type):
"""
This is a metaclass designed to act as an AbstractBaseClass.
You should rarely need to use this directly. Inheret from
the class (not metaclass) AbstractBaseClass instead.

Feel free to skip reading this metaclass and go on to the
documentation for AbstractBaseClass.
"""

def __init__(cls, name, bases, dict):
"""Initialize the class if Abstract requirements are met.

If the class is supposed to be abstract or it is concrete and
implements all @Abstract methods, then instantiate it.
Otherwise, an
AssertionError is raised.

Alternatively, if cls.__allow_abstract__ is True, then the
class is instantiated and no checks are done.
"""

if (__debug__ and not getattr(cls,'__allow_abstract__',False)
and not _AbstractMetaClass._IsSupposedToBeAbstract(cls)):
abstractMethods = _AbstractMetaClass._GetAbstractMethods(
cls.__bases__)
for name in abstractMethods:
if ( getattr(getattr(cls,name),'__abstract__',False)):
klasses =
_AbstractMetaClass._GetParentsRequiring(name,cls)
if (len(klasses)==0):
klasses = '(Unknown); all parents are %s.' % (
cls.__bases__)
else:
klasses = str(klasses)
raise AssertionError(
'Class %s must override %s to implement:\n%s.'
% (cls,name,klasses))

super(_AbstractMetaClass,cls).__init__(name,bases,dict)

def __call__(self, *args, **kw):
"""Only allow instantiation if Abstract requirements are met.

If there are methods that are still abstract and
__allow_abstract__
is not set to True, raise an assertion error. Otherwise,
instantiate the class.
"""
if (__debug__):
stillAbstract = [
name for name in
_AbstractMetaClass._GetAbstractMethods([self])
if (getattr(getattr(self,name),'__abstract__',False))]
assert (getattr(self,'__allow_abstract__',False)
or len(stillAbstract) == 0), (
"""Cannot instantiate abstract base class %s
because the follwoing methods are still abstract:\n%s""" %
(str(self),stillAbstract))
return type.__call__(self,*args,**kw)

def _IsSupposedToBeAbstract(cls):
"""Return true if cls is supposed to be an abstract class.

A class which is ''supposed to be abstract'' is one which
directly inherits from AbstractBaseClass. Due to metaclass
magic, the easiest way to check this is to look for the
__intended_abstract__ attribute which only AbstractBaseClass
should have.
"""
for parent in cls.__bases__:
if (parent.__dict__.get('__intended_abstract__',False)):
return True

@staticmethod
def _GetAbstractMethods(classList,abstractMethods=None):
"""Returns all abstract methods in a list of classes.

Takes classList which is a list of classes to look through and
optinally takes abstractMethods which is a dict containing
names
of abstract methods already found.
"""
if (None == abstractMethods):
abstractMethods = {}
for cls in classList:
for name in cls.__dict__:
method = getattr(cls,name)
if (callable(method) and
getattr(method,'__abstract__',False)):
abstractMethods[name] = True
_AbstractMetaClass._GetAbstractMethods(cls.__bases__,
   abstractMethods)
return abstractMethods.keys()

@staticmethod
def _GetParent