Confused about logger config from within Python (3)

[andrew cooke]

When I use a config file things seem to work (in other projects), but for my 
current code I hoped to configure logging from Python.

I distilled my problem down to the following test, which does not print 
anything.  Please can someone explain why?  I was expecting the module's logger 
to delegate to root, which has the DEBUG level set.

from logging import DEBUG, root, getLogger
from unittest import TestCase

class LoggingTest(TestCase):

def test_direct(self):
root.setLevel(DEBUG)
getLogger(__name__).debug("hello world")

Thanks,
Andrew
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Confused about logger config from within Python (3)

[andrew cooke]

similarly, if i run the following, i see only "done":

  from logging import DEBUG, root, getLogger

  if __name__ == '__main__':
  root.setLevel(DEBUG)
  getLogger(__name__).debug("hello world")
  print('done')
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Confused about logger config from within Python (3)

[andrew cooke]

On Friday, 28 December 2012 21:56:46 UTC-3, Peter Otten  wrote:
> Other revolutionary ideas: read the docs 
> 
>  ;)

how do you think i knew about the root handler without reading the damn docs 
you condescending asshole?

anyway, thanks for the help.

andrew

-- 
http://mail.python.org/mailman/listinfo/python-list

ordering with duck typing in 3.1

[andrew cooke]

hi,

please, what am i doing wrong here?  the docs say 
http://docs.python.org/release/3.1.3/library/stdtypes.html#comparisons "in 
general, __lt__() and __eq__() are sufficient, if you want the conventional 
meanings of the comparison operators" but i am seeing

>   assert 2 < three
E   TypeError: unorderable types: int() < IntVar()

with this test:


class IntVar(object):

def __init__(self, value=None):
if value is not None: value = int(value)
self.value = value

def setter(self):
def wrapper(stream_in, thunk):
self.value = thunk()
return self.value
return wrapper

def __int__(self):
return self.value

def __lt__(self, other):
return self.value < other

def __eq__(self, other):
return self.value == other

def __hash__(self):
return hash(self.value)


class DynamicTest(TestCase):

def test_lt(self):
three = IntVar(3)
assert three < 4
assert 2 < three
assert 3 == three

so what am i missing?

thanks,
andrew
-- 
http://mail.python.org/mailman/listinfo/python-list

python3.2m installed as (additional) binary

[andrew cooke]

Hi,

I just downloaded, built and altinstalled Python3.2 on Linux x64.  I noticed 
that in /usr/local/bin I have two identical (says diff) binaries called 
Python3.2 and Python3.2m.  Is this expected?  I can find very little reference 
to them apart from a short discussion in python-dev where someone asks if users 
will be confused by this and Barry Warsaw replies saying that they won't see 
them (well, I do!).

All I did was the usual ./configure; make; sudo make altinstall

So is this a bug?
Andrew

pl6 Python-3.2: ls -l /usr/local/bin/python3.2*
-rwxr-xr-x 2 root root 7368810 2011-02-27 13:03 /usr/local/bin/python3.2
-rwxr-xr-x 2 root root 7368810 2011-02-27 13:03 /usr/local/bin/python3.2m
-rwxr-xr-x 1 root root1826 2011-02-27 13:03 /usr/local/bin/python3.2m-config
pl6 Python-3.2: diff  /usr/local/bin/python3.2 /usr/local/bin/python3.2m
pl6 Python-3.2:

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python3.2m installed as (additional) binary

[andrew cooke]

[Sorry I clicked the wrong button so I think my prev reply went only to Tom]

Thanks.  Yes, they're hard linked.  And the bug report mentions PEP 3149 which 
says that "m" means --with-pymalloc was used 
http://www.python.org/dev/peps/pep-3149/

Cheers,
Andrew
-- 
http://mail.python.org/mailman/listinfo/python-list

Why is return type in getfullspec().annotations named as "return"?

[andrew cooke]

This conflicts with any parameter named "return".  Wouldn't it have been better 
to use "->" as the key?  Is there any way this can be changed?

Andrew
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why is return type in getfullspec().annotations named as "return"?

[andrew cooke]

Sorry, ignore that.  I just realised that "return" will be a reserved word, so 
that can't happen.  Andrew
-- 
http://mail.python.org/mailman/listinfo/python-list

Why is __root checked for in OrderedDict?

[andrew cooke]

If you look at the code in 
http://hg.python.org/cpython/file/6adbf5f3dafb/Lib/collections/__init__.py#l49 
the attribute __root is checked for, and only created if missing.  Why?

I ask because, from what I understand, the __init__ method will only be called 
when the object is first being created, so __root will always be missing.

My only guess is that this allows subclasses to do strange things without 
breaking the code (and if so, is a nice defensive coding pattern).  But I am 
worried I am missing something.

Thanks,
Andrew
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why is __root checked for in OrderedDict?

[andrew cooke]

Is that normal?  I mean, OK, it's possible (and yes I forgot it could be called 
directly), but is there any usual reason to do so?

I guess what I'm asking is: if I'm writing library code should I be this 
careful?   (I've written quite a lot of Python code without this ever biting 
me, but maybe I'm just lazy).  For example, would you consider it a bug if 
someone complained that calling __init__() resulted in unexpected behaviour?

Thanks,
Andrew
-- 
http://mail.python.org/mailman/listinfo/python-list

Confused about __prepare__

[andrew cooke]

In the code below I use __prepare__ to change the class dictionary so that a 
tuple is stored in __setitem__().  Since __getitem__() removes the tuple I 
wasn't expecting any problems, but it seems that __init__ is being retrieved 
via some other mechanism.  Why?  Is a copy of the dict being made somewhere?  
If so, can I change that?

Thanks,
Andrew


class TupleDict(dict):
'''Stores additional info, but removes it on __getitem__().'''

def __setitem__(self, key, value):
print('setting', key, value)
super(TupleDict, self).__setitem__(key, (value, 'secret'))

def __getitem__(self, key):
value = super(TupleDict, self).__getitem__(key)
print('getting', key, value[0]) # drop secret
return value[0]


class TupleMeta(type):

@classmethod
def __prepare__(metacls, name, bases, **kargs):
print('in prepare')
return TupleDict()

def __new__(cls, name, bases, classdict):
print('in new')
return type.__new__(cls, name, bases, classdict)


class C(metaclass=TupleMeta):

def __init__(self):
self.a = 1


c = C()

in prepare
setting __module__ __main__
setting __init__ 
in new
Traceback (most recent call last):
  File ..., line 34, in 
c = C() # TypeError: 'tuple' object is not callable
TypeError: 'tuple' object is not callable


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Confused about __prepare__

[andrew cooke]

Sorry I should probably have made clear that this is Python 3.2
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Confused about __prepare__

[andrew cooke]

Yes, I think you're right, thanks.  Makes sense from an efficiency POV.  
Luckily, it turns out I don't need to do that anyway :o)

Cheers,
Andrew
-- 
http://mail.python.org/mailman/listinfo/python-list

Replacing *instance* dict

[andrew cooke]

Related to the above, Is there anything wrong with the following code to 
replace the *instance* rather than the class dict?  It seems very crude, but 
appears to work.

Thanks,
Andrew


class TupleSuper:

def __new__(cls):
print('in new')
instance = object.__new__(cls)
instance.__dict__ = TupleDict(instance.__dict__)
return instance


class D(TupleSuper):

def __init__(self):
self.a = 1


if __name__ == '__main__':
d = D()
assert d.a == 1
d.a = 2
assert d.a == 2
d.a = 'three'
assert d.a == 'three'
print('woop')



On Thursday, April 7, 2011 7:31:16 PM UTC-3, andrew cooke wrote:
> 
> class TupleDict(dict):
> '''Stores additional info, but removes it on __getitem__().'''
> 
> def __setitem__(self, key, value):
> print('setting', key, value)
> super(TupleDict, self).__setitem__(key, (value, 'secret'))
> 
> def __getitem__(self, key):
> value = super(TupleDict, self).__getitem__(key)
> print('getting', key, value[0]) # drop secret
> return value[0]
-- 
http://mail.python.org/mailman/listinfo/python-list

My stupidity / strange inconsistency overriding class methods

[andrew cooke]

Hi,

I've been staring at this problem, in various forms, all day.  Am I missing 
something obvious, or is there some strange hardwiring of isinstance?  This is 
with Python 3.2.

class A(metaclass=ABCMeta):
@classmethod
def __instancecheck__(cls, instance): return False
# no override
assert isinstance(A(), A)
assert A.__class__.__instancecheck__(A, A())

class B(type):
def foo(self): return 42
class C(metaclass=B):
@classmethod
def foo(cls): return 7
# override
assert C().__class__.foo() == 7

It seems to me that the above two cases are inconsistent.  ABCMeta declares 
__instancecheck__ just like B declares foo.  Yet C can override foo, but A is 
unable to override the instance check.

Please help!

Thanks,
Andrew
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: My stupidity / strange inconsistency overriding class methods

[andrew cooke]

Also, there's something strange about the number of arguments (they're not 
consistent between the two examples - the "A" to __instancecheck__ should not 
be needed).  Yet it compiles and runs like that.  Very confused :o(
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: My stupidity / strange inconsistency overriding class methods

[andrew cooke]

OK, sorry, I see the mistake.  I'm confusing __class__ on the instance and on 
te class (the latter being the metaclass).  Sorry again, Andrew
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: My stupidity / strange inconsistency overriding class methods

[andrew cooke]

Thanks for finding that reference in the data model docs!  I was about to post 
a bug report because in PEP 3119 it says otherwise:

> The primary mechanism proposed here is to allow overloading the built-in 
> functions isinstance() and issubclass(). The overloading works as follows: 
> The call isinstance(x, C) first checks whether C.__instancecheck__ exists, 
> and if so, calls C.__instancecheck__(x) instead of its normal implementation. 

http://www.python.org/dev/peps/pep-3119/

But that's now what's implemented in Issue http://bugs.python.org/issue1708353 
which behaves as you quoted (only on the metaclass).

Cheers,
Andrew
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: My stupidity / strange inconsistency overriding class methods

[andrew cooke]

I didn't phrase that very well.  I do see the point about this being "an 
instance lookup on a class"...
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: meteclasses 2.x/3.x compatibility

[andrew cooke]

What I do in Lepl is use two stages.  The first calls the type/metaclass 
directly and the second subclasses that.  This avoids using the "sugar" that 
changes between 2 and 3.

So, for example, in 
http://code.google.com/p/lepl/source/browse/src/lepl/matchers/matcher.py#40 I 
have

  _Matcher = ABCMeta('_Matcher', (object, ), {})

and then

  class Matcher(_Matcher):
  ...

Andrew
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Language & lib reference in man format ?

[andrew cooke]

(1) Python's docs use Sphinx, which uses restructured text as a markup.  You 
can generate man pages from restructured text using rst2man (which is installed 
on my computer, probably as part of python/docutils).

HOWEVER I imagine it's not going to work very well, if at all, because Sphinx 
uses lots of fancy extensions.  But...

(2) Sphinx itself has a "builder" for man pages.  It's described at 
http://sphinx.pocoo.org/builders.html#sphinx.builders.manpage.ManualPageBuilder 
 So you should be able to configure Sphinx to use that.  So you will need to 
download the docs package, install Sphinx, and then tweak the Sphinx 
configuration as described at the link above and at 
http://sphinx.pocoo.org/config.html#confval-man_pages

This second approach should work.

Andrew
-- 
http://mail.python.org/mailman/listinfo/python-list

ABC-registered Exceptions are not caught as subclasses

[andrew cooke]

This isn't hugely surprising, but doesn't seem to be documented.  Is it a bug, 
or worth raising as one, or have I misunderstood?


Python 3.2 (r32:88445, Feb 27 2011, 13:00:05) 
[GCC 4.5.0 20100604 [gcc-4_5-branch revision 160292]] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from abc import ABCMeta
>>> class RootException(Exception,metaclass=ABCMeta): pass
... 
>>> class MyException(Exception): pass
... 
>>> RootException.register(MyException)
>>> try:
... raise MyException
... except RootException:
... print('caught')
... 
Traceback (most recent call last):
  File "", line 2, in 
__main__.MyException

If you assume that the ABC "register" class should work likeinheritance (as it 
does with issubclass and isinstance then you would, I think, have expected the 
exception above to have been caught.

Andrew
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: ABC-registered Exceptions are not caught as subclasses

[andrew cooke]

http://bugs.python.org/issue12029
-- 
http://mail.python.org/mailman/listinfo/python-list

Basic misunderstanding on object creation

[andrew cooke]

Hi,

The following code worked on Python 3.2, but no longer works in 3.4.  Did 
something change, or have I always been doing something dumb?

(I realise the code is pointless as is - it's the simplest example I can give 
of a problem I am seeing with more complex code).

>>> class Foo:
... def __new__(cls, *args, **kargs):
... print('new', args, kargs)
... super().__new__(cls, *args, **kargs)
... 
>>> class Bar(Foo):
... def __init__(self, a):
... print('init', a)
... 
>>> Bar(1)
new (1,) {}
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 4, in __new__
TypeError: object() takes no parameters

What I was expecting to happen (and what happens in 3.2) is that the 
object.__new__ method passes the argument to the __init__ of the subclass.

Any help appreciated.

Thanks,
Andrew
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Basic misunderstanding on object creation

[andrew cooke]

On Wednesday, 13 May 2015 11:36:12 UTC-3, Thomas Rachel  wrote:
> Am 13.05.2015 um 15:25 schrieb andrew cooke:
> 
> >>>> class Foo:
> > ... def __new__(cls, *args, **kargs):
> > ... print('new', args, kargs)
> > ... super().__new__(cls, *args, **kargs)
> 
> > new (1,) {}
> > Traceback (most recent call last):
> >File "", line 1, in 
> >File "", line 4, in __new__
> > TypeError: object() takes no parameters
> 
> object's __new__() dosn't take any parameters. So call it without arguments:
> 
> class Foo:
>  def __new__(cls, *args, **kargs):
>  print('new', args, kargs)
>  super().__new__(cls)
> 
> (at least if we know that we inherit from object. Might be that this one 
> doesn't work very good with multiple inheritance...)
> 
> 
> Thomas

But then nothing will be passed to __init__ on the subclass.

Andrew
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Basic misunderstanding on object creation

[andrew cooke]

> But then nothing will be passed to __init__ on the subclass.
> 
> Andrew

>>> class Foo:
... def __new__(cls, *args, **kargs):
... print('new', args, kargs)
... super().__new__(cls)
... 
>>> class Bar(Foo):
... def __init__(self, a):
... print('init', a)
... 
>>> Bar(1)
new (1,) {}

no "init" is printed.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Basic misunderstanding on object creation

[andrew cooke]

On Wednesday, 13 May 2015 11:56:21 UTC-3, Ian  wrote:
> On Wed, May 13, 2015 at 8:45 AM, andrew cooke  wrote:
> >>>> class Foo:
> > ... def __new__(cls, *args, **kargs):
> > ... print('new', args, kargs)
> > ... super().__new__(cls)
> > ...
> >>>> class Bar(Foo):
> > ... def __init__(self, a):
> > ... print('init', a)
> > ...
> >>>> Bar(1)
> > new (1,) {}
> >
> > no "init" is printed.
> 
> You're not returning anything from Foo.__new__, so the result of the
> constructor is None.  None.__init__ does nothing.

ah, you're right, thanks.  that was a typo.


more generally, it seems that the error is:

  (1) __new__ is called and then __init__ from some other, external code

  (2) in 3.2 anything passed to object's __new__ was silently discarded.

the following code works in 3.2 and 3.4:

class Foo:
def __new__(cls, *args, **kargs):
print("new", args, kargs)
return super().__new__(cls)

class Bar(Foo): 
def __init__(self, *args, **kargs):
print("init", args, kargs)

Bar(1)

(while my original code didn't).

thanks everyone,
andrew
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Basic misunderstanding on object creation

[andrew cooke]

On Wednesday, 13 May 2015 13:37:23 UTC-3, Terry Reedy  wrote:
> On 5/13/2015 9:25 AM, andrew cooke wrote:
> 
> > The following code worked on Python 3.2, but no longer works in 3.4.
> 
> Bugfixes break code that depends on buggy behavior. See
> https://bugs.python.org/issue1683368
> Your code also fails in 2.7.9 if you inherit Foo from object.
> The exact error messages changed for 3.4 in
> https://bugs.python.org/issue7963
> 
>  > Did something change,
> 
> Obviously yes.

thanks, but why does someone on this group always have to be a dick and make 
some smart-assed comment like this?

>  > or have I always been doing something dumb?
> 
> You were depending on behavior of object that Guido decided was buggy.
> 
> I found the tracker issue by looking for 'object' in the Core and 
> Builtins sections of the changelog one can access from What's New, first 
> paragraph (using Highlight All in Firefox).
> 
> >>>> class Foo:
> > ... def __new__(cls, *args, **kargs):
> > ... print('new', args, kargs)
> > ... super().__new__(cls, *args, **kargs)
> > ...
> >>>> class Bar(Foo):
> > ... def __init__(self, a):
> > ... print('init', a)
> > ...
> >>>> Bar(1)
> > new (1,) {}
> > Traceback (most recent call last):
> >File "", line 1, in 
> >File "", line 4, in __new__
> > TypeError: object() takes no parameters
> >
> > What I was expecting to happen (and what happens in 3.2) is that the 
> > object.__new__ method passes the argument to the __init__ of the subclass.
> 
> -- 
> Terry Jan Reedy

-- 
https://mail.python.org/mailman/listinfo/python-list

ABCs, functions, and __call__ (Python3)

[andrew cooke]

I think I'm missing something obvious here, so apologies in advance.

I'd like to be able to test whether something is a function or
implements __call__.  Now obviously I can do that as two separate
tests, but I though this was what ABCs were for.  However, for the
life of me I cannot find what the correct ABC would be.  Does it
exist?  Is there some other, more traditional approach I should be
using?  Or do I have to roll my own somehow?

Thanks for any guidance,
Andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: *Advanced* Python book?

[andrew cooke]

not direct answers, but

reading through the recipes can be interesting -
http://code.activestate.com/recipes/langs/python/

also, reading any good computing book and then wondering how you can
do that in python can help shed a new light on things.

andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: ABCs, functions, and __call__ (Python3)

[andrew cooke]

On Jan 16, 10:13 pm, Christian Heimes  wrote:
> Any callable in Python 3.0 has a "__call__" attribute.

Aha!  Thanks!  Andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pythonic way to handle coordinates

[andrew cooke]

although james's idea is quite neat (especially the idea of heritable
classes), my initial reaction was that it's too over-engineered.  so
here's a different approach.

there are two "tricks" that can help make using tuples easier:

(1) unpacking the return value.
  (x, y) = returns_a_point()
  x += 1

(2) passing a tuple to a function that takes an x and a y value:
  def takes_two_values(x, y):
...
  p = (1, 2)
  takes_two_values(*p)

and what I have done in the past is exploit these, so that functions/
methods that only takes a point takes two values while those that take
many points, other arguments, etc, take a tuple.  this sounds messy
but works quite well because you tend to use the first kind of
functions when you are changing coordinates (and then it is useful to
treat a point as separate x and y values) and the second kind of
functions when you are handling collections of points that don't need
modifying.

andrew
--
http://mail.python.org/mailman/listinfo/python-list

Relax Syntax for Augmented Arithmetic?

[andrew cooke]

Context - http://docs.python.org/3.0/reference/datamodel.html?highlight=data
model#object.__iadd__

Just a suggestion I thought I'd throw out...  There's a restriction in
the language implementation on exactly what can go the left of an
augmented arithmetic expression.

For example:
>>> a = 3
>>> a **= 2

is ok, but:
>>> class Foo():
...   def __init__():
... self.a = 3
...   def __ipow__(self, x):
... self.a **= x
...
>>> Foo() **= 2
  File "", line 1
SyntaxError: illegal expression for augmented assignment

Now unless I've done something stupid above (always a possibility :o)
the implementation seems a bit strict (is it really a *syntax* error?
- I am not sure exactly what the restriction is).

This may seems like a small issue, but operators can really help with
making embedded DSLs in Python - they give quite a bit of "wiggle
room" to invent a syntax that is compact and intuitive.  The
restriction above cuts into that (OK, so it's still a small
issue... :o)

Cheers,
Andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Relax Syntax for Augmented Arithmetic?

[andrew cooke]

> Therefore, Python requires you to rewrite the code in some other way
> that makes your intentions more clear. For instance, why not use the
> << operator instead?

Right, but you're guessing what the context is.  Within a DSL it often
makes a lot of sense to use operators for reasons that weren't
originally intended.

You even make the same case yourself indirectly.  The same argument
you make could be made to say that << should only operator on values
that can be shifted.  Now thankfully there is no way to test for that,
so there is no restriction and, consequently, it is now widely
accepted that no-one (even people arguing the case for constraints!)
think it odd to use << for something other than its initial use.

Obviously this kind of discussion has gone on since languages were
first invented - it's the "how much rope" argument.  So rather than
continue down that road I would just like to say that this feels like
an inconsistency.  The other operators are *not* as restricted and
this is making my life harder.

It may sound crazy, but this may force me to use * and ** instead (the
context is a language feature related to *args and **kargs, so the *
and ** help convey the meaning).  And they have a much much stronger
meaning to users, which will make my DSL harder to understand.  So in
this case a blunt knife is making life harder.

Andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: what's the point of rpython?

[andrew cooke]

Since this is a PyPy bashing thread, maybe it's an appropriate place
to suggest that the project has got a little bit waylaid by exploring
cool things instead of releasing a useful final result?

I am not questioning rpython directly - the case for something like
that is obvious.  But there's a question of balance.  It's possible to
go on building ever more complex systems which are theoretically
justified, but which postpone ever finishing the job.  At some point
there has to be a "good enough".

To some extent I am playing devil's advocate here, but as an outside
who looked at PyPy a while back, my uninformed and naive impression
was that the project was suffering from the kid of issues I have
caricatured above

Andrew

PS I guess you are aware of worse is better etc?  I think this may
also be a US/Euro culture issue...
--
http://mail.python.org/mailman/listinfo/python-list

Re: Relax Syntax for Augmented Arithmetic?

[andrew cooke]

Not sure if you were saying this, but the underlying technical reason
for this issue is that they are treated as assignment rather than
operators in the language spec -
http://docs.python.org/3.0/reference/simple_stmts.html#augmented-assignment-statements

I think this explains why they are not listed in the operator
precedence table http://docs.python.org/3.0/reference/expressions.html#summary

I think that's unfortunate in a language with mutable objects, but it
makes the decision seem much less arbitrary...

Cheers,
Andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Relax Syntax for Augmented Arithmetic?

[andrew cooke]

On Jan 18, 9:01 am, Chris Rebert  wrote:
> Indeed. Python happens to in this case draw the line at using the
> augmented assignment operators for non-assignment. I personally see
> this as reasonable because the = symbol has a consistent meaning in
> Python (assignment) whereas the other plain operators, as you bring
> up, consistently have no predetermined meaning;

my argument was that *= is not treated as = and *, but as a completely
new operator (the docs even say that the implementation need not
return self which suggests some pretty extreme semantics were
envisaged).  however, as i've just commented elsewhere, this
commitment to operators was only half-baked because they are parsed as
assignments.

anyway, to reply to your comment - *= is not predetermined.  it is
determined by __imul__ which is user-definable.

> but I do agree that it
> is in a sense an arbitrary restriction, like many programming language
> design choices. However, Python was not explicitly designed for
> creating DSLs,

python is a general programming language.  as far as i can make any
sense at all of your argument it seems to be "you are asking for
change, but this is not how the current system works".  to which the
obvious answer is: if it did work that way i wouldn't be asking for
change.

andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Relax Syntax for Augmented Arithmetic?

[andrew cooke]

On Jan 18, 9:40 am, Marc 'BlackJack' Rintsch  wrote:
> On Sun, 18 Jan 2009 04:24:04 -0800, andrew cooke wrote:
> > my argument was that *= is not treated as = and *, but as a completely
> > new operator (the docs even say that the implementation need not return
> > self which suggests some pretty extreme semantics were envisaged).
>
> What do you mean by "suggests … extreme semantics"?  Most natural thing
> is to use numbers and there you *have* to be able to return something
> different than `self` to get anything useful.  For instance:
>
>  n *= 3
>
> with `n` bound to a number different from zero can't return `self` from
> `__imul__`.

in your example, n is not a number, it is a mutable variable, and its
value changes.

when n is an instance implementing __imul__ the natural analogue is
that the internal state of the instance changes.

either i have misundertstood you, or you have misunderstood __imul__,
or you are treating = as equality, or maybe you are thinking of a pure
language that creates new instances?  python is impure.

anyway, my original request is moot.  i was assuming that this was a
"capricious" restriction.  in fact it's related to what i thought were
operators actually being assignments, and so no change is possible
(until python 4.0 when guido will finally see the light and move to s-
expressions, at which point everyone will stop using the language ;o)

andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Relax Syntax for Augmented Arithmetic?

[andrew cooke]

On Jan 18, 9:56 am, andrew cooke  wrote:

> either i have misundertstood you

ah, i see your point.  sorry, andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Relax Syntax for Augmented Arithmetic?

[andrew cooke]

http://bugs.python.org/issue4986

Sorry for the noise,
Andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Relax Syntax for Augmented Arithmetic?

[andrew cooke]

Improved link - 
http://docs.python.org/3.0/reference/datamodel.html#object.__iadd__
--
http://mail.python.org/mailman/listinfo/python-list

Re: function argument dependent on another function argument?

[andrew cooke]

>     sentinel = object()
>     ...
>
>     def foo(x, y=sentinel):
>       if y is sentinel:
>           y = self.a

it just struck me you could also do:

 def foo(self, x, *y_args)
   y = y_args[0] if y_args self.a

which more directly checks whether an argument was passed, but has the
downside of making the method signature less clear in the declaration.

andrew
--
http://mail.python.org/mailman/listinfo/python-list

Porting 3.0 to 2.6 - from __future__ import super missing?

[andrew cooke]

Hi,

I have some 3.0 code, which I would like to make work with 2.6.
However, there does not seem to be support for the new super() (no
args) via __future__.  Is that correct?  If so, what's the best way to
handle this?

Thanks,
Andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Porting 3.0 to 2.6 - from __future__ import super missing?

[andrew cooke]

On Jan 24, 10:39 am, Benjamin Peterson  wrote:
> andrew cooke  acooke.org> writes:
>
>
>
> > Hi,
>
> > I have some 3.0 code, which I would like to make work with 2.6.
> > However, there does not seem to be support for the new super() (no
> > args) via __future__.  Is that correct?  If so, what's the best way to
> > handle this?
>
> Just use the two argument super(): super(MyClass, instance) It's supported in
> both versions.

Thanks.  Any idea how to deal with ABCs?  It's sufficient to use a
simple class, but I want to expose an ABC in 3.0 as it will make it
easier for others to extend.

Unfortunately, "metaclass=" is a syntax error in 2.6 so the following
still fails:

from sys import version

if version.startswith('2.'):
class Matcher():
pass
else:
class Matcher(metaclass=ABCMeta):
pass

Andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Porting 3.0 to 2.6 - from __future__ import super missing?

[andrew cooke]

On Jan 24, 2:32 pm, Benjamin Peterson  wrote:
> I would suggest that you use the 2.6 syntax, and run "2to3 -f metaclass" on 
> your
> code. (ABCs have been backported to 2.6.)

Thanks - with that hint I found this -
http://mikewatkins.ca/2008/11/29/python-2-and-3-metaclasses/#using-the-metaclass-in-python-2-x-and-3-x

Andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: SimpleXMLRPCServer question

[andrew cooke]

On Jan 30, 11:59 pm, flagg  wrote:
> I am working on a very basic xmlrpc server, which will expose certain
> functions for administering BIND zone files.  The big problem I am
> having is parsing the incoming xmlrpc request.  Basically part of the
[...]

at the risk of repeating what the other guy said, it sounds like
you're going about this in the wrong way.  the xmlrpc server will
parse the incoming request for you if you supply it with a suitable
function to invoke.  so, for normal use, you do not override anything.

andrew

--
http://mail.python.org/mailman/listinfo/python-list

Where to host a (Python) project?

[andrew cooke]

Hi,

I have a new project, that I just released in beta (http://
www.acooke.org/lepl - a recursive decent parser with full
backtracking).  At the moment I am using pypi and setuptools for
distribution (it's a pure python package) and I am happy with hosting
static web pages (the manual and api doc linked to above) on my own
site (I did try the packages.python.org web pages, but it seemed
pointless duplicating my own).

However, i am thinking I could really do with:
- a mailing list
- simple bug tracking
- subversion
and am wondering which is the best (free) provider for these (the code
is LGPL open source).  I'd prefer a mailing list to something like
google groups (although I guess it may be possible to configure a
gateway) and I could open up my personal subversion server, but that
seems like a lot of work (not really that interested in moving to
something other than svn).

Any recommendations?

Thanks,
Andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Why do operators and methods of built-in types differ

[andrew cooke]

On Jan 31, 8:51 am, Csaba Hoch  wrote:
> What is the reason behind this difference between the __add__ operator
> and int.__add__?

this is quite common in python.  the special methods like __add__ are
used to implement some functionality (like '+' in this case), but they
are not all of it.  for example, when
  a + b
is evaluated, a.__add__(b) is attempted, but if that fails (raises a
NotImplemented error) then b.__radd__(a) is tried instead.

so there's not a 1-to-1 correspondence between '+' and __add__() and
that is reflected in the exceptions, too.  when a method does not
exist a NotImplemented error is raised, but '+' contains extra logic
and raises a more useful error message.

does that make sense?  i should probably add that this is just how i
understand things - i assume it's correct, but i've not looked
anything up in the documentation.

andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Why do operators and methods of built-in types differ

[andrew cooke]

> Just a correction: according to the doc, NotImplemented is not an
> error, but a returned value.

curious, so it is.  i wonder why there is both a special return value
(NotIMplemented) and a related exception (NotImplementedError).  seems
very odd to have a value...

andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Where to host a (Python) project?

[andrew cooke]

On Jan 31, 9:59 am, Martin  wrote:
> There's tigris.org, savannah (savannah.gnu.org, nongnu.org),
> launchpad. All of them are fine to some extent, you might want to read
> up on PyMotW about how Doug Hellmann decided where to host his stuff.

all i can find is that he is writing his own!
http://blog.doughellmann.com/search/label/codehosting
(his reqs are quite different to mine - I am looking for an external
provider because I do not want to host dynamic pages myself).

anyway, if that was what you mean, fine - i just wonder if i am
missing something?

thanks,
andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Where to host a (Python) project?

[andrew cooke]

On Jan 31, 11:22 am, eliben  wrote:
> code.google.com provides all of these in a free and convenient manner.
> Recommended.

unfortunately google don't seem that reliable ;o)  (have you tried a
google search today?)
--
http://mail.python.org/mailman/listinfo/python-list

Re: Where to host a (Python) project?

[andrew cooke]

On Jan 31, 4:50 pm, "Giampaolo Rodola'"  wrote:
> Google Code.
>
> --- Giampaolohttp://code.google.com/p/pyftpdlib

thanks - that's a nice example.  i'm a bit concerned about the whole
google corporation thing, but reading through the ideological check-
sheet at savannah convinced me i wasn't worthy and your project looks
good (i admit i haven't seen that many google projects, but they all
seemed abandoned/bare/hostile).  so i'll follow the majority here and
give google code a go.

cheers,
andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: help me python

[andrew cooke]

On Jan 31, 5:36 pm, aolsu...@gmail.com wrote:
> C:\Python26>vnc.py
> Traceback (most recent call last):
>   File "C:\Python26\vnc.py", line 4, in 
>     import PyD3DES
> ImportError: DLL load failed: The specified module could not be found.

i'm surprised no-one has replied here.  what is happening is that the
vnc module is trying to load a library it needs, called PyD3DES (this
is a compiled library, a ".dll", not python source).  a little
googling shows that this is part of the vnc package.

so either the vnc package is not installed correctly, or you have a
problem with paths.

since you are using windows i can't help you in any more detail than
that - hope it helps.

andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: database wrapper ?

[andrew cooke]

> Is SQLalchemy the best / most popular database wrapper ?

SQLAlchemy is the best SQL library I have ever used.

But it may depend on who you ask.  For me, what makes SQLAlchemy so
good is the way it allows you to use SQL from within Python.  I have
used the ORM side, and that's fine, but it's the way that it doesn't
get in the way of "just" SQL that make it so good.  If I had to choose
a second reason why it's so good it would be the way it emphasises
metadata - it will use table definitions from the database, and it
will let you define schema yourself (in fact, it's an excellent way of
defining complex schema in a platform-independent fashion).

The main drawback is that it is rather complex.  If something doesn't
work, it can be tricky to work out why.  On the other hand, the
documentation is good, and the support (on google groups) is good too
(a developer always replies within 24 hours in my experience).

It does expect you to know SQL - it doesn't try to hide SQL at all.
Whether that is good or bad depends on your POV I guess.

I wish all DB solutions were like this - my hope is that EmpireDB will
do the same for Java, but it's too early to tell...

Andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Where to host a (Python) project?

[andrew cooke]

On Feb 1, 8:45 pm, a...@pythoncraft.com (Aahz) wrote:
> Note that it's fairly easy to get a new list hosted at python.org, just
> ask postmaster.  I for one won't participate in any list hosted on
> Google because of the need for a Google login.

ah well - i guess you can use pyparsing ;o)

http://code.google.com/p/lepl/

andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: database wrapper ?

[andrew cooke]

> I wish all DB solutions were like this - my hope is that EmpireDB will
> do the same for Java, but it's too early to tell...

Hmmm - I should correct the above.  I had assumed EmpireDB was new,
because it's an Apache Incubator project, but now I look at their site
I see it's actually been around for years.  I need to try it out...

Sorry about the error,
Andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Where to host a (Python) project?

[andrew cooke]

On Feb 1, 8:45 pm, a...@pythoncraft.com (Aahz) wrote:
> [...] I for one won't participate in any list hosted on
> Google because of the need for a Google login.

hi,  just fyi, i investigated this and you can join any publicly
readable group by sending an email to the "-subscribe" address.  you
do not need a google login for this and, as far as i can tell, it then
operates for you like a normal mailing list.

for example, to subscribe to the group foo, you would send an email to
foo-subscr...@googlegroups.com.  to unsubscribe, use foo-
unsubscr...@googlegroups.com.

this isn't exactly well-publicised, but i tested it and it does work.

andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: what IDE is the best to write python?

[andrew cooke]

> Just to register a contrary opinion: I *hate* syntax highlighting

you can use it in surprising ways, if the implementation is good
enough.  when i used intellij with java i had the "syntax"
highlighting set so that everything was simple black+white, but
mutable state (i can't remember the exact details; perhaps it was just
variables that were assigned, i am not sure what happened with
attributes) was in bold.  very neat - showed where the likely bugs
were in the code - and nothing to do with what you'd normally consider
syntax highlighting...

andrew

(last time i looked, intellij's support for python was not very good.
which is a pity, because it was an excellent ide - better than
eclipse, imho).
--
http://mail.python.org/mailman/listinfo/python-list

Re: Locating python

[andrew cooke]

sorry, you are using easy_install, so

  sudo easy_install 

instead of what i said a moment ago.  the important thing is to use
"sudo".

andrew

On Feb 3, 7:30 pm, andrew cooke  wrote:
> the exact details of what you are reporting seem a bit odd, but i have
> seen a similar error because i have tried to use my own account to
> install the package, instead of using root.
>
> what i believe happens is that easy_install first tries to create the
> "test" file, and then checks it is there.  if it is not there, you get
> an error.  normally the error explains that you need to run as root,
> but i assume in your case it doesn't - that seems odd, but it seemed
> worth mentioning anyway.
>
> so if you have not done so, instead of
>
>   python setup.py install
>
> do
>
>   sudo python setup.py install
>
> more generally, i have used opensuse 11.0 and 11.1, and generally
> things work just fine, so i would check you are following all the
> instructions correctly.
>
> good luck,
> andrew
>
> On Feb 3, 6:24 pm, David Sevilla  wrote:
>
> > [...]
> > [Errno 2] No such file or directory: '/usr/local/lib/python2.5/site-
> > packages/test-easy-install-3728.pth'
>
> > The installation directory you specified (via --install-dir, --prefix,
> > or the distutils default setting) was:
>
> >     /usr/local/lib/python2.5/site-packages/
>
> > This directory does not currently exist. [...]
>
> > From what I have gathered by reading here and there, it seems that the
> > actual path for site-packages is not the place where it is being
> > looked for. Sure enough, I have /usr/local/lib/python2.5/site-
> > packages/ . What worries me is that there is no file called test*
>
>

--
http://mail.python.org/mailman/listinfo/python-list

Re: Locating python

[andrew cooke]

the exact details of what you are reporting seem a bit odd, but i have
seen a similar error because i have tried to use my own account to
install the package, instead of using root.

what i believe happens is that easy_install first tries to create the
"test" file, and then checks it is there.  if it is not there, you get
an error.  normally the error explains that you need to run as root,
but i assume in your case it doesn't - that seems odd, but it seemed
worth mentioning anyway.

so if you have not done so, instead of

  python setup.py install

do

  sudo python setup.py install

more generally, i have used opensuse 11.0 and 11.1, and generally
things work just fine, so i would check you are following all the
instructions correctly.

good luck,
andrew

On Feb 3, 6:24 pm, David Sevilla  wrote:
> [...]
> [Errno 2] No such file or directory: '/usr/local/lib/python2.5/site-
> packages/test-easy-install-3728.pth'
>
> The installation directory you specified (via --install-dir, --prefix,
> or the distutils default setting) was:
>
>     /usr/local/lib/python2.5/site-packages/
>
> This directory does not currently exist. [...]
>
> From what I have gathered by reading here and there, it seems that the
> actual path for site-packages is not the place where it is being
> looked for. Sure enough, I have /usr/local/lib/python2.5/site-
> packages/ . What worries me is that there is no file called test*
--
http://mail.python.org/mailman/listinfo/python-list

Re: parse date/time from a log entry with only strftime (and no regexen)

[andrew cooke]

> > ValueError: unconverted data remains:  this is the remainder of the log  
> > line
> > that I do not care about

you could catch the ValueError and split at the ':' in the .args
attribute to find the extra data.  you could then find the extra data
in the original string, use the index to remove it, and re-parse the
time.

ugly, but should work.
andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Locating python

[andrew cooke]

On Feb 3, 7:35 pm, David Sevilla  wrote:
> I am quite new to Linux, and thought that by using yast2 there would
> be no user problems (I am asked for the root password). I will sudo it
> to see if it solves the problem.

yast asked you for the password so that easy_install could be
installed correctly.

you are now using "sudo easy_install" to install mnemosyne and sudo is
asking for the root password.

each time you install something you need to change public files on the
system, and so each time you need to use root in some way.  yast does
this by logging in as root for you, but needs the password to do it.
sudo does the same thing, but again needs the password to do it.

hope that makes senses (and that this worked).

andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Locating python

[andrew cooke]

On Feb 4, 9:16 am, andrew cooke  wrote:
> > actually "su" needs the root (or the target users') password
> > and sudo needs _your_ (the current users) password.
>
> argh, sorry for the confusion.

actually, no.  sudo requires the root password.  at least on opensuse
11.1 default config.  i just tried it:

> Python-2.5.4: sudo make install
root's password:

this is explained in the sudoers file:

 # In the default (unconfigured) configuration, sudo asks for the root
password.
 # This allows use of an ordinary user account for administration of a
freshly
 # installed system. When configuring sudo, delete the two
 # following lines:
 Defaults targetpw   # ask for the password of the target user i.e.
root
 ALL ALL=(ALL) ALL   # WARNING! Only use this together with
'Defaults targetpw'!

andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Locating python

[andrew cooke]

> actually "su" needs the root (or the target users') password
> and sudo needs _your_ (the current users) password.

argh, sorry for the confusion.
--
http://mail.python.org/mailman/listinfo/python-list

Structuring Modules with a Ubiquitous Base Class (Circular Dependencies)

[andrew cooke]

Is there a good solution to the following problem?

I have a library whose components I would like to separate into
distinct modules.  These components inherit from a common base class
that provides common functionality (the inheritance is important only
for implementation; there's a separate ABC mechanism for typing).

Now as that stands, there is no problem.  Each module could import the
common base class.

Unfortunately the base class itself references many of the components
(the reason for this is that supplies operator implementations (like
__add__) which are shortcuts for the components themselves).

This leads to a circular dependency - the base class wants to import
the components, which in turn want to import the base class.

Is there any standard solution to this?

Thanks,
Andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Structuring Modules with a Ubiquitous Base Class (Circular Dependencies)

[andrew cooke]

On Feb 4, 7:49 pm, andrew cooke  wrote:
> This leads to a circular dependency - the base class wants to import
> the components, which in turn want to import the base class.
>
> Is there any standard solution to this?

well, to partially answer my own question, this is certainly
possible.  in the general case it might get quite complex, but for the
structure described it is quite simple.  the 'trick' is to import the
component in the *method* of the common base class.

for example:

class Base(object):

def __init__(self):
from first import First
from second import Second
self.first = lambda *args: First(*args)
self.second = lambda *args: Second(*args)

where First, defined in first, subclasses Base in the normal way.

however, i suspect this will have a performance hit, since "linking"
is being done at "run time" rather than "compile time".  anyone have
any guidance on how serious that would be?  i guess it could be
ameliorated by doing the work in the constructor (as above, which is
just that way for a compact example - in "real life" the components
are used in a more complex manner).

and i still suspect there is a more efficient metaclass or similar
approach.

andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to find wxPython method documentation??

[andrew cooke]

On Feb 4, 8:06 pm, len  wrote:
> How does one find the methods that are available in the classes.

heh.  welcome to the wonderful world of wxpython :o(

if you use eclipse to edit your code, then (providing the wind is in
the right direction and the file you are editing doesn't have any
syntax errors) pressing F3 when you are on a particular method will
take you to the definition.  or, at least, one of the definitions with
that name.

and if you want to see a list of available methods the simplest way is
to use tab completion (or dir() in python itself).

if you're not using eclipse (with pydev) check to see if the ide/
editor you are using has something similar.

also, wxpython comes with some examples, all packaged in a demo
program.  that is your best source of documentation.  go through all
the examples in there and look at the code (the demo program will show
you the code and even let you edit it and see the results of your
changes).

to be honest, wxpython is a bit of a nightmare (imho). but the results
can be worth it.

good luck,
andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Comparing two book chapters (text files)

[andrew cooke]

On Feb 4, 10:20 pm, Nick Matzke  wrote:
> So I have an interesting challenge.  I want to compare two book
> chapters, which I have in plain text format, and find out (a) percentage
> similarity and (b) what has changed.

no idea if it will help, but i found this yesterday - http://www.nltk.org/

it's a python toolkit for natural language processing.  there's a book
at http://www.nltk.org/book with much more info.

andrew
--
http://mail.python.org/mailman/listinfo/python-list

Re: Ordered dict by default

[andrew cooke]

so what is happening with pep 372?

http://www.python.org/dev/peps/pep-0372/
--
http://mail.python.org/mailman/listinfo/python-list

Re: "Weird" Indentation? (Or: is there a for...else construct?)

[andrew cooke]

there's a justification for this awful mess here -
http://mail.python.org/pipermail/python-3000/2006-March/000104.html

i didn't know about this, and even after reading steven's broken (i
assume) example, managed to get it backwards.

the else is if there *isn't* a break and is for search loops (see link
above).

(it's still in 3).

andrew


Steven D'Aprano wrote:
> Andreas Waldenburger wrote:
>
>> It seems that there is a for...else construct. Replacing the inner if
>> with pass seems to confirm this. The else clause is still executed.
>
> Yes, there is a for...else construct.
>
> The else block runs if the for loop exits *without* a break.
>
> for i in range(20):
> if i == 10: break
> else:
> print "no break here"
>
> for i in range(20):
> if i == 100: break
> else:
> print "no break here"
>
>
>> What's broken here: Python or my brain?
>
> Perhaps we should not answer that question.
>
>
> --
> Steven
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>


--
http://mail.python.org/mailman/listinfo/python-list

Re: Python3.0 has more duplication in source code than Python2.5

[andrew cooke]

Steve Holden wrote:
>> You'd best hope the copied section was thoroughly reviewed otherwise
>> you're
>> duplicating a flaw across X other sections. And then you also best hope
>> that
>> whoever finds said flaw and fixes it is also smart enough to check for
>> similar constructs around the code base.
>>
> This is probably preferable to five different developers solving the
> same problem five different ways and introducing three *different* bugs,
> no?

someone posted some numbers that suggested that more code than normal was
copied in python 3.0.  that seems reasonable, as others have said, because
it's a new major release.  but as far as i know, this is the first time
it's been raised.  so it seems like a useful piece of information that
might help improve python in some way.  which should be welcomed.

yet the general tone of the responses has been more defensive than i would
have expected.  i don't really understand why.  nothing really terrible,
given the extremes you get on the net in general, but still a little
disappointing.

the email quoted above is a typical example.  as i said - nothing
terrible, just a misleading false dichotomy.  yes, five people solving it
five different ways would be worse, but that doesn't mean there isn't some
better solution.  surely it would be preferable if there was one way, that
didn't involve copying code, that everyone could use?

i'm not saying there is such a solution.  i'm not even saying that there
is certainly a problem.  i'm just making the quiet observation that the
original information is interesting, might be useful, and should be
welcomed.

andrew


--
http://mail.python.org/mailman/listinfo/python-list

Re: generator object or 'send' method?

[andrew cooke]

If I were experimenting with Python to see just how far I could push
coroutines at the moment, I would use .send() and look at how I could
factor things into a small library (containing, for example, your
trap-and-response secondary generator).

But if this was paid work, I would write a class with a __next__ method
and do things explicitly.

In answer to your PS, via a roundabout route: I've done something similar
recently by using .throw() to raise an exception in the body of the
generator.  This was done with a reset() function.  So, for example:

  mygen = ...
  a = next(mygen)
  b = next(mygen)
  reset(mygen)
  c = next(mygen)

And in the generator:

  while True:
try:
  ...
  yield ...
catch ResetException:
  yield # discarded by the reset function

And finally:

  def reset(gen):
gen.throw(ResetException())

In that case, I needed an extra "yield" that did nothing.

Andrew


Aaron Brady wrote:
> Hello,
>
> I am writing a generator to return a sequence of numbers with some
> variation.  The parameters of the variation can be changed by the
> caller, even after the generator is started.  My question is, is it
> better to wrap the generator in an object, so that the parameters can
> be changed just by an attribute access?  Or keep the generator clear,
> and modify parameters with a 'send' call?
>
> P.S.  It would receive the 'send' from a different point in control
> flow than its usual 'next'.  Should it repeat a value if it receives a
> 'send'?  Or should I wrap it in a secondary 'trap_send_and_repeat'
> generator?
>
> Thanks sincerely as always.
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>


--
http://mail.python.org/mailman/listinfo/python-list

Re: generator object or 'send' method?

[andrew cooke]

steven probably knows this, but to flag the issue for people who are
looking at generators/coroutines for the first time: there's a little
"gotcha" about exactly how the two sides of the conversation are
synchronized.  in simple terms: send also receives.

unfortunately the example steven gave doesn't really show this, so i've
modified it below.  you can now see that the first next() after .send()
receives 2, not 1.  note that i am using python 3, so the .next() method
is .__next__() (the asymmetry between next and send seems odd to me, but
there you go).

(in a sense this is completely logical, but if you're used to the
asynchronous way the internet works, it can seem unintuitive)

how to handle this was one of the things the original post was asking
about (if i understood correctly).

>>> def gen(n):
...   while True:
... obj = yield n
... n += 1
... if obj is not None: n = obj
...
>>> g = gen(5)
>>> next(g)
5
>>> g.__next__()
6
>>> g.send(1)
1
>>> next(g)
2

andrew


Steven D'Aprano wrote:
> On Tue, 10 Feb 2009 05:28:26 +, John O'Hagan wrote:
>> I would love to see a simple code example of this if you have one; I've
>> been wanting to do this but couldn't even get started.
>
> Is this too simple?
>
 def gen(n):
> ... while True:
> ... obj = yield n
> ... if obj is not None: n = obj
> ...
 g = gen(5)
 g.next()
> 5
 g.next()
> 5
 g.send(12)
> 12
 g.next()
> 12


--
http://mail.python.org/mailman/listinfo/python-list

Re: Convert date/time to unix timestamp?

[andrew cooke]

the python routines are a bit basic - you really have to think quite hard
about what you are doing to get the right answer.

in your case, you need to be clear what the timezone is for the datetime
you are using.  timezone info is optional (see the datetime documentation,
where it talks about "naive" and "aware" objects).

anyway, ignoring that, i think you need the very useful, but oddly
located, calendar.timegm().

andrew


Phillip B Oldham wrote:
> Is there a simple way to set a date/time and convert it to a unix
> timestamp? After some googling I found the following:
>
> t = datetime.time(7,0,0)
> starttime = time.mktime(t.timetuple())+1e-6*t.microsecond
>
> That seems like very long-winded. Is there an easier way? I've read
> the docs on the datetime and time modules, but nothing's jumping out
> at me.
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>


--
http://mail.python.org/mailman/listinfo/python-list

Re: Functional schmunctional...

[andrew cooke]

r0g wrote:
> def ip2inet(a):
>   li = a.split('.')
>   assert len(li) == 4 or len(li) == 6
>   return reduce(add,[int(li[e])*(256**((len(li)-1)-e)) for e in
> xrange(0,len(li))])

what a mess.

i don't use this extreme a functional style in python (it's not really how
the language is intended to be used), but i think you can do better than
that.

how about:

from itertools import count

def ip2inet(a):
blocks = a.split('.')
assert len(blocks) in (4, 6)
return sum(map(lambda (i, n): int(i) * 256**n,
   zip(reversed(blocks), count(0

i haven't looked at the other function, but as a general comment it sounds
me like you are in such a hurry to point out fp is bad that you haven't
bothered to master it first.

andrew


--
http://mail.python.org/mailman/listinfo/python-list

Re: Avoiding argument checking in recursive calls

[andrew cooke]

Terry Reedy wrote:
> Reverse the test order
>
> def fact(n):
>  if n > 0: return fact(n-1)*n
>  if n == 0: return 1
>  raise ValueError

sweet!  but is this generally possible?  ie: did you think this up for
this question or is it an idiom that you find yourself using often?

andrew

--
http://mail.python.org/mailman/listinfo/python-list

Re: Another optimization request :-)

[andrew cooke]

why are you dong this point by point?  surely you can express the physics
as a set of equations and invert the matrix?  wouldn't that be a lot
faster?  you'd replace the iteration over all combinations of points with
a faster matrix inversion.

see for example
http://www.medwelljournals.com/fulltext/ajit/2006/324-338.pdf page 331
onwards.

there's a very nice into to the verlet integration mentioned here -
http://teknikus.dk/tj/gdc2001.htm

andrew


jeffg wrote:
> If anyone wants to take this on... I would really really like to have
> the spring_layout modified to support multi-threading if at all
> possible.
> My test data is 20,000, which makes this process 20,000 x 20,000 or
> 400,000,000 (400 million) calculations.  This is taking somewhere
> between 2-3 hours an iteration.
> I plan to plot out over 1,000,000 data points or more, which would put
> this at 1,000,000,000,000 (1 trillion) calculations.  Any help in
> making this more efficient would be much appreciated.
>
> def spring_layout(G, iterations=50, dim=2, node_pos=None,
> verbose=False):
> """Spring force model layout"""
> if node_pos==None :  # set the initial positions randomly in 1x1
> box
> vpos=random_layout(G, dim=dim)
> else:
> vpos=node_pos
> if iterations==0:
> return vpos
> if G.order()==0:
> k=1.0
> else:
> k=N.sqrt(1.0/G.order()) # optimal distance between nodes
> disp={} # displacements
>
> # initial "temperature" (about .1 of domain area)
> # this is the largest step allowed in the dynamics
> # linearly step down by dt on each iteration so
> # on last iteration it is size dt.
> t=0.1
> dt=0.1/float(iterations+1)
> for i in range(0,iterations):
> for v in G:
> if verbose==True:
> print("Interation: " + str(i + 1) + ", Calculating: "
> + str(v.encode('iso-8859-15', "replace")))
> disp[v]=N.zeros(dim)
> for u in G:
> delta=vpos[v]-vpos[u]
> dn=max(sqrt(N.dot(delta,delta)),0.01)
> # repulsive force between all
> deltaf=delta*k**2/dn**2
> disp[v]=disp[v]+deltaf
> # attractive force between neighbors
> if G.has_edge(v,u):
> deltaf=-delta*dn**2/(k*dn)
> disp[v]=disp[v]+deltaf
>
> # update positions
> for v in G:
> l=max(sqrt(N.dot(disp[v],disp[v])),0.01)
> vpos[v]=vpos[v]+ disp[v]*t/l
> t-=dt
> return vpos
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>


--
http://mail.python.org/mailman/listinfo/python-list

Re: Another optimization request :-)

[andrew cooke]

sorry, that was stupid.  there is no inversion (apart from 1/m), just the
integration.

still, improving the integration would allow larger steps.

andrew



andrew cooke wrote:
>
> why are you dong this point by point?  surely you can express the physics
> as a set of equations and invert the matrix?  wouldn't that be a lot
> faster?  you'd replace the iteration over all combinations of points with
> a faster matrix inversion.
>
> see for example
> http://www.medwelljournals.com/fulltext/ajit/2006/324-338.pdf page 331
> onwards.
>
> there's a very nice into to the verlet integration mentioned here -
> http://teknikus.dk/tj/gdc2001.htm
>
> andrew
>
>
> jeffg wrote:
>> If anyone wants to take this on... I would really really like to have
>> the spring_layout modified to support multi-threading if at all
>> possible.
>> My test data is 20,000, which makes this process 20,000 x 20,000 or
>> 400,000,000 (400 million) calculations.  This is taking somewhere
>> between 2-3 hours an iteration.
>> I plan to plot out over 1,000,000 data points or more, which would put
>> this at 1,000,000,000,000 (1 trillion) calculations.  Any help in
>> making this more efficient would be much appreciated.
>>
>> def spring_layout(G, iterations=50, dim=2, node_pos=None,
>> verbose=False):
>> """Spring force model layout"""
>> if node_pos==None :  # set the initial positions randomly in 1x1
>> box
>> vpos=random_layout(G, dim=dim)
>> else:
>> vpos=node_pos
>> if iterations==0:
>> return vpos
>> if G.order()==0:
>> k=1.0
>> else:
>> k=N.sqrt(1.0/G.order()) # optimal distance between nodes
>> disp={} # displacements
>>
>> # initial "temperature" (about .1 of domain area)
>> # this is the largest step allowed in the dynamics
>> # linearly step down by dt on each iteration so
>> # on last iteration it is size dt.
>> t=0.1
>> dt=0.1/float(iterations+1)
>> for i in range(0,iterations):
>> for v in G:
>> if verbose==True:
>> print("Interation: " + str(i + 1) + ", Calculating: "
>> + str(v.encode('iso-8859-15', "replace")))
>> disp[v]=N.zeros(dim)
>> for u in G:
>> delta=vpos[v]-vpos[u]
>> dn=max(sqrt(N.dot(delta,delta)),0.01)
>> # repulsive force between all
>> deltaf=delta*k**2/dn**2
>> disp[v]=disp[v]+deltaf
>> # attractive force between neighbors
>> if G.has_edge(v,u):
>> deltaf=-delta*dn**2/(k*dn)
>> disp[v]=disp[v]+deltaf
>>
>> # update positions
>> for v in G:
>> l=max(sqrt(N.dot(disp[v],disp[v])),0.01)
>> vpos[v]=vpos[v]+ disp[v]*t/l
>> t-=dt
>> return vpos
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>>
>>
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>


--
http://mail.python.org/mailman/listinfo/python-list

Re: Another optimization request :-)

[andrew cooke]

jeffg wrote:
> To be honest, this is not my code and I'm new to python.  It's part of
> the open source project NetworkX, but I'm using this one call
> extensively.  I'm also not that familiar with the math behind the
> physics.  I'll read the documents and see if I can figure it
> out.  :-)  Thank you for the links and suggestions.  I really need to
> get this code performing at peak levels.

the maths is quite simple really, if you know any physics.  it imagines
that a spring connects each point, and then works out what the total
effect of all the springs is.  that gives the net "push" on each point. 
it then goes through and moves each point a small amount in the direction
of the push (there's a "max" that makes no sense physically but probably
gives better numeric stability).

once they have moved, their positions have changed, so everything needs to
be calculated again, hence the iteration.

terry's suggestion was that, rather than working particle by particle, if
you can generalise the code to use matrices then you can use numpy.  that
would mean more of the calculation is done using C rather than Python, and
so would be a big speed-up.

that means that you need to rewrite the program so that you don't have the
for-loops in Python (for u.. and for v...).  instead you need to call
numpy to do array calculations.

that is going to give you much more speedup than what i recommended
(modifying the integration step at the end).

to be honest, although this is not hard if you are used to this kind of
thing, if the above isn't obvious it is quite a job.  you would probably
be better looking for a different library.  unfortunately i don't know of
one (i looked for exactly this a month or two ago and concluded i would
have to write my own; i didn't have time so i used a simpler layout).  i'm
really busy, or i would do this for you, because it's interesting and
useful.  but the code you have looks "amateur" - it wasn't written by
someone who does numerical work for a living.  that doesn't mean it's not
correct or that the original author was stupid, but it does mean that to
make it efficient means looking at the problem in a different way.

there are certainly (very good) java libraries that do this.  could you
use jpython and call out to one of those?  if so, that's probably your
best approach.

andrew


--
http://mail.python.org/mailman/listinfo/python-list

Re: Spam

[andrew cooke]

A quick search on "imap nntp" turned up this list that might be useful -
http://deflexion.com/messaging/ although I wonder when it was written
because I remember using Aaron's RSS to email aggregator when RSS was
new(!).

It mentions gmane, though, which certainly still exists (I assume it
carries this list too).  And this page suggests you can read gmane via
nntp - http://reticule.gmane.org/

Aha!  yes!  It;s in the FAQ :o)

  Can I read news via secure NNTP (nntps)?
  Yes. Point your news reader towards nntps://snews.gmane.org/.

http://www.gmane.org/faq.php

I should be working; I will try that this evening.  What was the name of
the client that threaded messages with a cute ascii tree?!

Cheers,
Andrew


Tim Chase wrote:
> Though I guess this thread does beg the question:  is there a way
> to read the filtered mailing list with a newsreader (via NNTP)


--
http://mail.python.org/mailman/listinfo/python-list

Re: how to distribute python extensions independently of python

[andrew cooke]

You might want to read
https://www.dfwpython.org/repo/Presentations/2008-10-04-PyArkansas-PythonEggsIntro/eggs-introduction.pdf

It covers a lot of ground; I used it to work out how to distribute a pure
Python package, but I am sure it mentions compiled packages too.  In my
case I ended up using setuptools (easy_install) and distutils.

(Not sure if you asking just about compiling, or more general packaging,
but IIRC it covers both).

Andrew


Travis wrote:
> So,
>
> Recently I made a fix to the zlib module that I need for use at work.
>
> I would like other employees to be able to use it without recompiling
> python.
>
> I assume I can just rename it and distribute it as a python extension.
>
> I was wondering how I can provide a way for other employees to build it.
>
> I saw a "Universal Unix Makefile for Python extensions" that looks
> promising.
>
> Is this the accepted way to compile python extensions still?
> --
> Crypto ergo sum.  http://www.subspacefield.org/~travis/
> Do unto other faiths as you would have them do unto yours.
> If you are a spammer, please email j...@subspacefield.org to get
> blacklisted.
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>


--
http://mail.python.org/mailman/listinfo/python-list

Re: Spam

[andrew cooke]

Grant Edwards wrote:
>> I should be working; I will try that this evening.  What was the name of
>> the client that threaded messages with a cute ascii tree?!
>
> slrn?

i think i was remembering trn, which is now apparently dead.  will try
slrn...  thanks, andrew


--
http://mail.python.org/mailman/listinfo/python-list

Re: Problem with objects copying each other in memory

[andrew cooke]

you're setting the new knight's "sl" to the value self.sl and then adding
values to it.  that's the same list - so you are adding values to the
self.sl list when you add them to the knight's sl.

this is easier to understand just by seeing the fix, which is to use:

temp = Knight(self.x, self.y, self.g, self.h, self.gp, list(self.sl))

which will make a new copy of the list, so you are only adding to the sl
in the (new) knight.

andrew



dlocpuwons wrote:
> Using Python 2.6.1...
>
> I am (attempting) to make an A* search for a chess problem, but I am
> running into a really annoying shared memory issue in my successor
> function. Here it is stripped down to the important parts that relate
> to my problem.
>
> def successors(self):
>   result = []
>   moves = [[2, 1], [2, -1], [-2, 1], [-2, -1], [1, 2], [1, -2], 
> [-1,
> 2], [-1, -2]] #possible moves for a knight
>
>   for i in moves:
>   temp = Knight(self.x, self.y, self.g, self.h, self.gp, 
> self.sl)
>   temp.x += i[0]
>   temp.y += i[1]
>   temp.sl.append([temp.x, temp.y]) #Adds the new current 
> state to the
> visited states list
>   result.append(temp)
>   return result
>
> The method creates a temporary Knight object, increments it to the new
> position and then adds this new position to its list of visited
> states. Then it returns a list of these 8 new objects. The problem
> seems to be with the "result.sl.append()" line. As the method chugs
> along and creates the new objects the "temp.sl" lines seems to stay in
> memory, so when the method is done all the new objects have all the
> new states that were made over the course of the method instead of
> just their own created in the loop. For example when I try to get
> successors for a piece that is initially at (2,2) with no previously
> visited states, the method prints this out for the sl (state list)
> value
>
> [[2, 2], [4, 3], [4, 1], [0, 3], [0, 1], [3, 4], [3, 0], [1, 4], [1,
> 0]]
> [[2, 2], [4, 3], [4, 1], [0, 3], [0, 1], [3, 4], [3, 0], [1, 4], [1,
> 0]]
> [[2, 2], [4, 3], [4, 1], [0, 3], [0, 1], [3, 4], [3, 0], [1, 4], [1,
> 0]]
> [[2, 2], [4, 3], [4, 1], [0, 3], [0, 1], [3, 4], [3, 0], [1, 4], [1,
> 0]]
> [[2, 2], [4, 3], [4, 1], [0, 3], [0, 1], [3, 4], [3, 0], [1, 4], [1,
> 0]]
> [[2, 2], [4, 3], [4, 1], [0, 3], [0, 1], [3, 4], [3, 0], [1, 4], [1,
> 0]]
> [[2, 2], [4, 3], [4, 1], [0, 3], [0, 1], [3, 4], [3, 0], [1, 4], [1,
> 0]]
> [[2, 2], [4, 3], [4, 1], [0, 3], [0, 1], [3, 4], [3, 0], [1, 4], [1,
> 0]]
>
> but what it should print out is
>
> [[2, 2], [4, 3]]
> [[2, 2], [4, 1]]
> [[2, 2], [0, 3]]
> [[2, 2], [0, 1]]
> [[2, 2], [3, 4]]
> [[2, 2], [3, 0]]
> [[2, 2], [1, 4]]
> [[2, 2], [1, 0]]
>
> It sort of seems like python is trying to be too smart and is trying
> to keep things in memory. Is there anyway to work around this?
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>


--
http://mail.python.org/mailman/listinfo/python-list

Re: Problem with objects copying each other in memory

[andrew cooke]

the official answer to that question is here -
http://docs.python.org/reference/datamodel.html?highlight=containers - at
about the 6th paragraph where it talks about "containers".

i think it's a hard question and really in a sense (imho) the real reason
is just that this tends to be the best compromise for the kind of language
that python is.

andrew


Cameron Pulsford wrote:
> Thanks, that did it! Why is that the case though? Or rather, why do
> the assignments to temp.x and temp.y not effect the self.x and self.y?
> How come I only run into the problem with the list?
>
>
> On Feb 12, 2009, at 5:15 PM, andrew cooke wrote:
>
>>
>> you're setting the new knight's "sl" to the value self.sl and then
>> adding
>> values to it.  that's the same list - so you are adding values to the
>> self.sl list when you add them to the knight's sl.
>>
>> this is easier to understand just by seeing the fix, which is to use:
>>
>> temp = Knight(self.x, self.y, self.g, self.h, self.gp, list(self.sl))
>>
>> which will make a new copy of the list, so you are only adding to
>> the sl
>> in the (new) knight.
>>
>> andrew
>>
>>
>>
>> dlocpuwons wrote:
>>> Using Python 2.6.1...
>>>
>>> I am (attempting) to make an A* search for a chess problem, but I am
>>> running into a really annoying shared memory issue in my successor
>>> function. Here it is stripped down to the important parts that relate
>>> to my problem.
>>>
>>> def successors(self):
>>> result = []
>>> moves = [[2, 1], [2, -1], [-2, 1], [-2, -1], [1, 2], [1, -2], 
>>> [-1,
>>> 2], [-1, -2]] #possible moves for a knight
>>>
>>> for i in moves:
>>> temp = Knight(self.x, self.y, self.g, self.h, self.gp, 
>>> self.sl)
>>> temp.x += i[0]
>>> temp.y += i[1]
>>> temp.sl.append([temp.x, temp.y]) #Adds the new current 
>>> state to
>>> the
>>> visited states list
>>> result.append(temp)
>>> return result
>>>
>>> The method creates a temporary Knight object, increments it to the
>>> new
>>> position and then adds this new position to its list of visited
>>> states. Then it returns a list of these 8 new objects. The problem
>>> seems to be with the "result.sl.append()" line. As the method chugs
>>> along and creates the new objects the "temp.sl" lines seems to stay
>>> in
>>> memory, so when the method is done all the new objects have all the
>>> new states that were made over the course of the method instead of
>>> just their own created in the loop. For example when I try to get
>>> successors for a piece that is initially at (2,2) with no previously
>>> visited states, the method prints this out for the sl (state list)
>>> value
>>>
>>> [[2, 2], [4, 3], [4, 1], [0, 3], [0, 1], [3, 4], [3, 0], [1, 4], [1,
>>> 0]]
>>> [[2, 2], [4, 3], [4, 1], [0, 3], [0, 1], [3, 4], [3, 0], [1, 4], [1,
>>> 0]]
>>> [[2, 2], [4, 3], [4, 1], [0, 3], [0, 1], [3, 4], [3, 0], [1, 4], [1,
>>> 0]]
>>> [[2, 2], [4, 3], [4, 1], [0, 3], [0, 1], [3, 4], [3, 0], [1, 4], [1,
>>> 0]]
>>> [[2, 2], [4, 3], [4, 1], [0, 3], [0, 1], [3, 4], [3, 0], [1, 4], [1,
>>> 0]]
>>> [[2, 2], [4, 3], [4, 1], [0, 3], [0, 1], [3, 4], [3, 0], [1, 4], [1,
>>> 0]]
>>> [[2, 2], [4, 3], [4, 1], [0, 3], [0, 1], [3, 4], [3, 0], [1, 4], [1,
>>> 0]]
>>> [[2, 2], [4, 3], [4, 1], [0, 3], [0, 1], [3, 4], [3, 0], [1, 4], [1,
>>> 0]]
>>>
>>> but what it should print out is
>>>
>>> [[2, 2], [4, 3]]
>>> [[2, 2], [4, 1]]
>>> [[2, 2], [0, 3]]
>>> [[2, 2], [0, 1]]
>>> [[2, 2], [3, 4]]
>>> [[2, 2], [3, 0]]
>>> [[2, 2], [1, 4]]
>>> [[2, 2], [1, 0]]
>>>
>>> It sort of seems like python is trying to be too smart and is trying
>>> to keep things in memory. Is there anyway to work around this?
>>> --
>>> http://mail.python.org/mailman/listinfo/python-list
>>>
>>>
>>
>>
>
>


--
http://mail.python.org/mailman/listinfo/python-list

Re: sgmllib parser keeps old tag data?

[andrew cooke]

you are declaring class variables, not instance variables.  you need to
declare these in an __init__ method.  RTFM. 
http://docs.python.org/tutorial/classes.html#a-first-look-at-classes

Berend van Berkum wrote:
> class MyParser(sgmllib.SGMLParser):
> 
> content = ''
> markup = []
> span_stack = []


--
http://mail.python.org/mailman/listinfo/python-list

Re: sgmllib parser keeps old tag data?

[andrew cooke]

Sorry, this reply was delayed (trying to use usenet...) and so now seems
(even more) bad tempered than needed.  Andrew

andrew cooke wrote:
> you are declaring class variables, not instance variables.  you need to
> declare these in an __init__ method.  RTFM.
> http://docs.python.org/tutorial/classes.html#a-first-look-at-classes
> 
> Berend van Berkum wrote:
>> class MyParser(sgmllib.SGMLParser):
>> 
>> content = ''
>> markup = []
>> span_stack = []
> 
> 
> --
> http://mail.python.org/mailman/listinfo/python-list


--
http://mail.python.org/mailman/listinfo/python-list

Re: confusion about variable scope in a class

[andrew cooke]

it's not a scope issue.  you are confusing variables and objects.

a variable is a box that can hold an object

so x = 2 puts the object '2' in the box 'x'.

following that with x = '3' changes the box 'x' to hold the object '3'.

but lists are also boxes, different from variables.

so x = [1,2,3]

puts the a list object, that is a box that contains 1, 2 and 3, in 'x'

then y = x

puts THE SAME list object in box 'y'.

then y[1] = 4 changes the second item in the list's box to 4

the print x gives "[1,4,3]" because you have changed contents of the list.

in contrast x = 3 then y = x then y = 4 does not change x because you care
changing variables, not list contents.

to fix your code, you need to copy the list

x = [1,2,3]
y = list(x) # copy
y[1] = 4
print x
[1,2,3]

surely this is ina faq?  it comes up once a day...

andrew



gyro wrote:
> Hi,
> I was writing a Python script to perform some data analyses and was
> surprised by some behavior I noted. A simple test program illustrating
> the behavior is below.
> I do not understand why the value of 'data' is being modified. I am
> obviously missing something obvious, and would certainly appreciate an
> explanation of why this is happening.
>
> Thank you.
>
> -gf
>
> --
>
> #!/bin/env python
>
> class TestPop(object):
> def round1(self,data1):
> t = data1.pop(-1)
>
> def round2(self,data2):
> t = data2.pop(-1)
>
> def tester(self):
> data = range(10)
> self.round1(data)
> print data
> self.round2(data)
> print data
>
> if __name__ == '__main__':
> tp = TestPop()
> tp.tester()
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>


--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[andrew cooke]

rushen...@gmail.com wrote:
> Hi everybody,
> I am an engineer. I am trying to improve my software development
> abilities. I have started programming with ruby. I like it very much
> but i want to add something more. According to my previous research i
> have designed a learning path for myself. It's like something below.
>   1. Ruby (Mastering as much as possible)
>   2. Python (Mastering as much as possible)
>   3. Basic C++ or Basic Java
> And the story begins here. As i search on the net,  I have found that
> because of the natural characteristics of python such as GIL, we are
> not able to write multi threaded programs. Oooops, in a kind of time
> with lots of cpu cores and we are not able to write multi threaded
> programs. That is out of fashion. How a such powerful language doesn't
> support multi threading. That is a big minus for python. But there is
> something interesting, something like multi processing. But is it a
> real alternative for multi threading. As i searched it is not, it
> requires heavy hardware requirements (lots of memory, lots of cpu
> power). Also it is not easy to implement, too much extra code...

I understand why you are asking this question - you want to learn, and
that is good - but as you say, you are a beginner, and you are focussing
on one thing that has caught your eye when there are many other issues
that are more important.

The GIL is an implementation detail.  I suspect that it could be largely
removed if there was sufficient need.  But that still wouldn't make Python
a good language for programming on multiple cores.  That's not as big a
deal as you think, because we currently DON'T KNOW what would make a good
language for programming on multiple cores - it's an open topic in the
wider community.

It may be, for example, that the best approaches for concurrent
programming require a lot of automatic pre-processing that needs static
type information.  Or that mutable state is simply too much of a problem
and pure functional programming is the only way forwards.  Both of these
would be much more serious problems for Python than the GIL.  On the other
hand, Python has inbuilt support for co-routines.  Experience gained with
that might lead towards a more actors-like solution that fits naturally
within Python.

So my first point is that you are worrying about a problem that no-one yet
know how to solve, and worrying about a small and largely irrelevant part
of that.


Second, you are committing the common mistake of over-estimating the
importance of efficiency.  Python is not a fast language; it never has
been.  That does not stop it being extremely useful.  This is largely
because when it needs to do "hard work" it delegates to C libraries.  Why
can this not apply to concurrent programming too?  Maybe the top level
logic will stay in a single thread, because that is easier to program, but
libraries will use multiple cores.

So my second point is that you are being too restrictive in considering
what a future solution might look like.


In conclusion, then, I strongly suggest you stop worrying so much about
things that you don't yet have the experience to see completely, and
instead get more experience under your belt.  That sounds more harsh than
I really mean - obviously worrying about this kind of thing is part of
learning.

Incidentally, if you already know Ruby and want to improve your abilities
I am not sure learning Python is the best use of your time.  The two
languages are very similar.  You might be better taking a huge leap to
something like Haskell or OCaml.  Or, if you want to get hands-on
experience of concurrency now, Erlang.

Andrew



> After all of that, i start to think about omiting python from my
> carrier path and directly choosing c++ or java. But i know google or
> youtube uses python very much. How can they choose a language which
> will be killed by multi threading a time in near future. I like python
> and its syntax, its flexibility.
>
> What do you think about multi threading and its effect on python. Why
> does python have such a break and what is the fix. Is it worth to make
> investment of time and money to a language it can not take advantage
> of multi cores?
>
> Thank you...
> Rushen
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>


--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[andrew cooke]

andrew cooke wrote:
> something like Haskell or OCaml.  Or, if you want to get hands-on
> experience of concurrency now, Erlang.

I think for once I said something useful there.  I think you would
probably enjoy Erlang, and it would be very useful for understanding
concurrency.  Also, Erlang is not as strange a language as Haskell or
OCaml.  You will probably find it quite familiar.  And there's an
excellent book (Joe Armstrong's "Programming Erlang").

At the risk of hastening Python's demise :o) I strongly encourage you to
learn Erlang instead of Python.

Andrew


--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[andrew cooke]

why do you think that current work is ignorant of occam?  occam itself was
based on hoare's "communicating sequential processes" which is a classic
of the field.  the ideas behind occam are not unknown and it hasn't been
forgotten (there are many libraries based on synchronous message passing;
one for java called jcsp for example -
http://www.cs.kent.ac.uk/projects/ofa/jcsp/ ; the "rendezvous model"
(receiving tasks wait for messages) is used in ada).

but really it did very little to hide the complexities of parallel
computing - it's famous because it (and the transputer platform) was one
of the first languages to take parallelism "seriously", not because it
presented any kind of silver bullet (more generally, it was a pretty crude
language, more suited to small embedded applications than large projects -
it didn't even have dynamically sized arrays)

there's a comment here http://lambda-the-ultimate.org/node/2437 that shows
the limitations of occam: "I used Occam (the transputer implementation of
CSP) very heavily in the 1980s and early 1990s, and eventually started
referring to channels as "the return of the GOTO", since in any moderately
complex application, you spent a lot of time wondering, "If I put bytes in
*here*, who will they go to?" Addressable actors and/or tuple spaces both
felt much more scalable (in the coding sense)."

(disclaimer - i haven't used it personally.  once i was asked to maintain
an occam system, but somehow managed to dodge the responsibility)

if you look at erlang, which is one of the more successful parallel
languages at the moment, you'll see some similarity to occam (message
passing is explicit), but shifting to asynchronous messages helps give a
more robust system.

andrew


Hendrik van Rooyen wrote:
> "Aahz"  wrote:
>
>
>> In article ,
>> Hendrik van Rooyen  wrote:
>> >
>> >Occam was the language that should have won the marketing prize, but
>> >didn't.
>>
>> It wasn't simple enough.
>
> I thought (at the time) that it was quite good at hiding some
> horrible complexities of communication between different
> processes on the same, and different processors.
>
> All done by the compiler, automagically.
>
> I think now that a hard look at the underlying techniques
> used then could add something to the debate referred to
> earlier - but there may be a barrier because the dataflow
> or systolic array type programming model is not one
> that is currently fashionable.
>
> - Hendrik
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>


--
http://mail.python.org/mailman/listinfo/python-list

Re: Will multithreading make python less popular?

[andrew cooke]

Bruno Desthuilliers wrote:
> rushen...@gmail.com a écrit :
> (snip)
>> And the story begins here. As i search on the net,  I have found that
>> because of the natural characteristics of python such as GIL, we are
>> not able to write multi threaded programs.
>
> I'm surprised no one here corrected that point yet, so here we go: yes,
> Python does support multithreading. The fact that threads won't be
> executed concurrently on a multicore machine (due to the GIL) is a
> different problem (cf Andrew Cooke's answer on this - and his advice to
> study Erlang if you want to do concurrent programming).

ah, sorry, i noticed this last night in another comment from rushenaly,
but forgot to say anything.

in case it's still not clear: you can have threads even on a single core. 
this is done by "time slicing" - some cpu time is given to one thread,
then to another.

exactly who does the slicing can vary.  in the case of a gui (which is
what i was about to explain last night then got distracted) it's quite
common for the gui library itself to do the scheduling of work.  so even
though the gui library uses a single thread, it can update several
windows, handle user input, etc "in parallel".  the next level is that the
language itself does the scheduling - that's what is commonly called
"threads".  finally the operating system can share things out (and that's
called processes).  but these are all basically the same thing, can happen
on a single core, and are not affected by the GIL (i have simplified
above; threads can also be a service that the operating system provides to
a language)

andrew


--
http://mail.python.org/mailman/listinfo/python-list

Re: Is there something easier than ORM?

[andrew cooke]

Philip Semanchuk wrote:
> In short, I gather that others on this list are a lot more fond of
> SqlAlchemy and ORMs in general than I am. Granted, my experience is
> very limited. I tried to integrate SqlAlchemy in one project,
> struggled for a long time to express how I wanted my tables joined,
> and finally found that performance was bad compared to our homegrown
> SQL. My boss and I were both very comfortable with SQL and were happy
> to go back to writing our own SQL statements and coding a data access
> layer.
>
> I don't intend this as a criticism of SqlAlchemy. On the contrary I am
> impressed by what it does. But I often see people promoting ORM as the
> solution to all database access problems, and I certainly don't feel
> like it is.

the reason i, at least, like sqlalchemy so much is for exactly the reasons
you outlined.  unlike other orm solutions it doesn't force you to use orm.
 you can also use sql directly - either as simple strings or by
constructing it via python methods (which can be a very powerful way of
programatically constructing sql commands that would be a nightmare to
write by hand).  that gives you the flexibility to deal with each problem
in the way that feels most natural, without having to switch between tools
(in fact, i have mixed orm and "direct" sql in a single project with no
problems using sqlalchemy - reading from one database using sql and
writing to another using objects).

andrew


--
http://mail.python.org/mailman/listinfo/python-list

Re: number theory libraries / project euler

[andrew cooke]

eliben wrote:
> Hello,
>
> What are some good & recommended number theory libs for Python (or
> accessible interfaces to C libs), for things like primes,
> factorization, etc. Naturally, speed is of utmost importance here.

i just read the project site and one of the things they say on their front
page is that all problems have a solution that should run in "under a
minute".  the emphasis is on finding the right algorithm, not brute force
number crunching.  so i am not sure that speed is of much importance at
all.

> In other words, which Python libraries and tools to you use to help
> you solve Project Euler problems :-) ?

you may want to look at numpy and scipy, but looking at the first dozen
questions i think ordinary python may be sufficient.

andrew


--
http://mail.python.org/mailman/listinfo/python-list

Re: SVN/CVS and Branching

[andrew cooke]

maybe this is just me, but i don't have a clue what your problem is.  what
does "starting imports all over the place" mean?  what do you mean by
"retired"?

i use svn with python in exactly the same way as with java (and, i
thought, in the same way as anyone uses svn with any language; java uses
the directory structure as a package structure too).

maybe someone else will reply, but if not it might help to explain a
little more detail.

andrew


Jeff Dyke wrote:
> Hello.  I am curious about different ideas on how you handle branching
> your python projects.  With some other languages this is trivial, but
> since python uses these directories as modules and i have the top
> level module starting imports all over the place, i am wondering what
> others do.  In the past we had retired the first branch and just moved
> towards the new, but that is not possible now.


--
http://mail.python.org/mailman/listinfo/python-list

Re: SVN/CVS and Branching

[andrew cooke]

Ah, OK.  I have never worked like that.  The only helpful comment I can
think of is that I believe svn will store links correctly (while CVS
doesn't).

However, I do find the whole approach a bit odd.  You seem to be doing
your own versioning "by hand" (having two packages that are equivalent
with different names), where I would leave that to SVN/CVS.  So what I
would do, and what seems to me a more natural way to use version control,
is to branch the entire project (the whole tree), leave the foo module
name the same, and work on two separate projects.  If changes (ie on
modules other than foo) need to be made to both projects I would use
svnmerge to make sure that merges made to one also apply to the other. 
But if that is a common situation then it suggests the other modules might
be better treated as a separate library, with their own project.

On the other hand, svnmerge is a bit of a pain and I can see how your
approach might make more sense (particularly on smaller projects).

Andrew


Jeff Dyke wrote:
> Fair enough.  Say my project is called foo, and it has many
> submodules.  So there are imports that may look like `import foo.bar`
> or `from foo.bar import baz`, if i change the top level directory, it
> is no longer foo and then those imports do not work as originally
> written.  The way i currently do this  is to create a branch, say
> foo2, and create a symbolic link named foo pointing at foo2, after
> renaming foo, when i want to work on the branch and remove the link
> when i want to work on the head.  This actually works fine, but
> thought there may be a better way.
>
> Jeff
>
> On Wed, Feb 18, 2009 at 7:40 PM, andrew cooke  wrote:
>>
>> maybe this is just me, but i don't have a clue what your problem is.
>> what
>> does "starting imports all over the place" mean?  what do you mean by
>> "retired"?
>>
>> i use svn with python in exactly the same way as with java (and, i
>> thought, in the same way as anyone uses svn with any language; java uses
>> the directory structure as a package structure too).
>>
>> maybe someone else will reply, but if not it might help to explain a
>> little more detail.
>>
>> andrew
>>
>>
>> Jeff Dyke wrote:
>>> Hello.  I am curious about different ideas on how you handle branching
>>> your python projects.  With some other languages this is trivial, but
>>> since python uses these directories as modules and i have the top
>>> level module starting imports all over the place, i am wondering what
>>> others do.  In the past we had retired the first branch and just moved
>>> towards the new, but that is not possible now.
>>
>>
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>


--
http://mail.python.org/mailman/listinfo/python-list

Re: Porting to new Python version

[andrew cooke]

i don't know what the context is, so it's hard for me to comment on the
decision (i assume there are commerical pressures like customers not
wanting to install old versions).

however,if you go ahead, you need to think about exactly what you want to
target.

the latest version is really 3.0.1.  moving to 3 is probably not that hard
(and there are tools to automate the process).  also, 2.6 is largely
compatible with 3.  so moving to something that works with 2.6 and 3 is
probably a reasonable target.  but that code will not work, without more
significant effort, on 2.5 and earlier.

so one idea would be to keep your existing code for 2.4, and update for 3.
 for some time (years) you will need to support two versions, but if it is
stable then that may not be much work.

alternatively, you could ignore 3, which is not going to be mainstream for
some time, and simply run against 2.6 and 2.5.  that should be even less
work because you're staying with 2.  you could then say that your code
works on 2.4 through 2.6

or, finally, you could do the second step above (to get code that works on
2.4 to 2.6) and then the first step (to get a separate version that works
on 3).  doing things in that order (with staged releases) lets you get
most bug fixes for 2.5/6 into the code before branching for 3.

hope that makes sense.

disclaimer - i have not done the above; this is from reading various
newsgroups and attempting to backport a project written in 3 to 2 (it was
easy to go to 2.6, but i failed to get the same code to run on 2.5).

andrew





Gabor Urban wrote:
> Hi,
>
> I have a tough issue: we are using a Python application written quite
> a time ago for version 2.4. The code is mature, and there are no bugs.
>  My bosses came up with the idea to port it to the latest release... I
> am not really convinced that it's a good step.
>
> I wellcome any information pro and contra. I would like to get the
> background as precisely as possible.
>
> Thanks in advance and good day to You!
>
> Gabor
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>


--
http://mail.python.org/mailman/listinfo/python-list

Re: Porting to new Python version

[andrew cooke]

maybe i should clarify that "easy" below is going to be relative.  the
process may end up being very hard due to various other reasons.  what i
was trying to explain is that

(1) 3 is probably going to require a separate branch from 2;

(2) that 2.6 and 3 can both be considered "latest";

(3) moving from 2.4 to 2.6 is probably best done before branching for 3;

(4) moving from 2.4 to 2.6 is probably easier than moving from 2 to 3.

andrew


andrew cooke wrote:
>
> i don't know what the context is, so it's hard for me to comment on the
> decision (i assume there are commerical pressures like customers not
> wanting to install old versions).
>
> however,if you go ahead, you need to think about exactly what you want to
> target.
>
> the latest version is really 3.0.1.  moving to 3 is probably not that hard
> (and there are tools to automate the process).  also, 2.6 is largely
> compatible with 3.  so moving to something that works with 2.6 and 3 is
> probably a reasonable target.  but that code will not work, without more
> significant effort, on 2.5 and earlier.
>
> so one idea would be to keep your existing code for 2.4, and update for 3.
>  for some time (years) you will need to support two versions, but if it is
> stable then that may not be much work.
>
> alternatively, you could ignore 3, which is not going to be mainstream for
> some time, and simply run against 2.6 and 2.5.  that should be even less
> work because you're staying with 2.  you could then say that your code
> works on 2.4 through 2.6
>
> or, finally, you could do the second step above (to get code that works on
> 2.4 to 2.6) and then the first step (to get a separate version that works
> on 3).  doing things in that order (with staged releases) lets you get
> most bug fixes for 2.5/6 into the code before branching for 3.
>
> hope that makes sense.
>
> disclaimer - i have not done the above; this is from reading various
> newsgroups and attempting to backport a project written in 3 to 2 (it was
> easy to go to 2.6, but i failed to get the same code to run on 2.5).
>
> andrew
>
>
>
>
>
> Gabor Urban wrote:
>> Hi,
>>
>> I have a tough issue: we are using a Python application written quite
>> a time ago for version 2.4. The code is mature, and there are no bugs.
>>  My bosses came up with the idea to port it to the latest release... I
>> am not really convinced that it's a good step.
>>
>> I wellcome any information pro and contra. I would like to get the
>> background as precisely as possible.
>>
>> Thanks in advance and good day to You!
>>
>> Gabor
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>>
>>
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>


--
http://mail.python.org/mailman/listinfo/python-list

Re: Regular expression bug?

[andrew cooke]

i wonder what fraction of people posting with "bug?" in their titles here
actually find bugs?

anyway, how about:

re.findall('[A-Z]?[a-z]*', 'fooBarBaz')

or

re.findall('([A-Z][a-z]*|[a-z]+)', 'fooBarBaz')

(you have to specify what you're matching and lookahead/back doesn't do
that).

andrew


Ron Garret wrote:
> I'm trying to split a CamelCase string into its constituent components.
> This kind of works:
>
 re.split('[a-z][A-Z]', 'fooBarBaz')
> ['fo', 'a', 'az']
>
> but it consumes the boundary characters.  To fix this I tried using
> lookahead and lookbehind patterns instead, but it doesn't work:
>
 re.split('((?<=[a-z])(?=[A-Z]))', 'fooBarBaz')
> ['fooBarBaz']
>
> However, it does seem to work with findall:
>
 re.findall('(?<=[a-z])(?=[A-Z])', 'fooBarBaz')
> ['', '']
>
> So the regular expression seems to be doing the Right Thing.  Is this a
> bug in re.split, or am I missing something?
>
> (BTW, I tried looking at the source code for the re module, but I could
> not find the relevant code.  re.split calls sre_compile.compile().split,
> but the string 'split' does not appear in sre_compile.py.  So where does
> this method come from?)
>
> I'm using Python2.5.
>
> Thanks,
> rg
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>


--
http://mail.python.org/mailman/listinfo/python-list

Re: code challenge: generate minimal expressions using only digits 1,2,3

[andrew cooke]

this is a neat problem.

here is what i would do: use generators that extend an input.  a stream
approach.  the initial input would be the numbers themselves.

[('1', 1),('2', 2),('3', 3)]

those are (expression, value) pairs

then an initial attempt at the next function would be to extend that list
with additions:

def additions(pairs):
  for (expr1, value1) in pairs:
# first, pass through unchanged
yield (expr1, value1)
# then generate all possible additions
for (expr2, value2) in pairs:
  yield ('%s+%s'%(value1, value2), value1 + value2))

this would give you:

[('1', 1),('2', 2),('3', 3), ('1+1', 2), ...]

(you may need to add parentheses to expressions to preserve meaning
correctly)

you could extend that with an extra loop over different operations.
(subtraction, multiplication, etc)

then you could repeat that as often as you want (eating its own tail, in a
sense, i think).  an infinite list is ok because these are generators.

then you could filter that to group expressions that give a certain value,
and find the shortest.

andrew



Trip Technician wrote:
> anyone interested in looking at the following problem.
>
> we are trying to express numbers as minimal expressions using only the
> digits one two and three, with conventional arithmetic. so for
> instance
>
> 33 = 2^(3+2)+1 = 3^3+(3*2)
>
> are both minimal, using 4 digits but
>
> 33 = ((3+2)*2+1)*3
>
> using 5 is not.
>
> I have tried coding a function to return the minimal representation
> for any integer, but haven't cracked it so far. The naive first
> attempt is to generate lots of random strings, eval() them and sort by
> size and value. this is inelegant and slow.
>
> I have a dim intuition that it could be done with a very clever bit of
> recursion, but the exact form so far eludes me.
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>


--
http://mail.python.org/mailman/listinfo/python-list

Re: code challenge: generate minimal expressions using only digits 1,2,3

[andrew cooke]

repeat(compose(ints,
# all_operators)))(START))
# clipping values to below 1000 makes things much faster
# note 200,000 below, 20,000 above!
summarise(compose(take(20),
      repeat(compose(ints, small,
 all_operators)))(START))
# get to larger values faster by sorting in the repeat
#sort = lambda x: sorted(x, key=value, reverse=True)
#summarise(compose(take(20),
#  repeat(compose(ints, small,
# all_operators),
# preproc=sort))(START))



andrew cooke wrote:
>
> this is a neat problem.
>
> here is what i would do: use generators that extend an input.  a stream
approach.  the initial input would be the numbers themselves.
>
> [('1', 1),('2', 2),('3', 3)]
>
> those are (expression, value) pairs
>
> then an initial attempt at the next function would be to extend that
list
> with additions:
>
> def additions(pairs):
>   for (expr1, value1) in pairs:
> # first, pass through unchanged
> yield (expr1, value1)
> # then generate all possible additions
> for (expr2, value2) in pairs:
>   yield ('%s+%s'%(value1, value2), value1 + value2))
>
> this would give you:
>
> [('1', 1),('2', 2),('3', 3), ('1+1', 2), ...]
>
> (you may need to add parentheses to expressions to preserve meaning
correctly)
>
> you could extend that with an extra loop over different operations.
(subtraction, multiplication, etc)
>
> then you could repeat that as often as you want (eating its own tail, in
a
> sense, i think).  an infinite list is ok because these are generators.
>
> then you could filter that to group expressions that give a certain
value,
> and find the shortest.
>
> andrew
>
>
>
> Trip Technician wrote:
>> anyone interested in looking at the following problem.
>> we are trying to express numbers as minimal expressions using only the
digits one two and three, with conventional arithmetic. so for
>> instance
>> 33 = 2^(3+2)+1 = 3^3+(3*2)
>> are both minimal, using 4 digits but
>> 33 = ((3+2)*2+1)*3
>> using 5 is not.
>> I have tried coding a function to return the minimal representation for
any integer, but haven't cracked it so far. The naive first attempt is
to generate lots of random strings, eval() them and sort by size and
value. this is inelegant and slow.
>> I have a dim intuition that it could be done with a very clever bit of
recursion, but the exact form so far eludes me.
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>




--
http://mail.python.org/mailman/listinfo/python-list