[Python-Dev] script containing all opcs

[Michal Revucky]

hello everyone,

i would like to get a python script which executes all interpreter's opcodes, or
how am i supposed to create such script... i just need to make sure that all
opcodes (as defined in Include/opcode.h) are executed by this scrip 
i need this script for testing purposes while rewritting the python's ceval.c 
code...

bye michal
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] script containing all opcs

[Georg Brandl]

Michal Revucky schrieb:
> hello everyone,
> 
> i would like to get a python script which executes all interpreter's opcodes, 
> or
> how am i supposed to create such script... i just need to make sure that all
> opcodes (as defined in Include/opcode.h) are executed by this scrip 
> i need this script for testing purposes while rewritting the python's ceval.c 
> code...

This question might better suit comp.lang.python, but nevertheless:
The opcodes are quite well documented, in the documentation of the
"dis" module. Many of them serve a specific purpose (like implementing
`x[y]` or `raise x`) and should therefore be easy to test. For the
others, like ROT_TWO, you'll need to look into Python/compile.c to
find out in which cases they are generated.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Documentation Error for __hash__

[Ondrej Certik]

> I strongly advise people interested in this topic to take a closer look
> at the functionality that was added to address issue 2235. The "don't
> inherit __hash__" behaviour has been backported from 3.0 and broke a
> bunch of code, but the naive fix of reverting to the 2.5 behaviour
> proceeded to make the 2.6 version of collections.Hashable completely
> useless (since pretty much *everything* then claimed to be hashable,
> including all of the container types in the standard library).
>
> After thrashing out those problems, the 2.6 behaviour ended up being:
> - __hash__ is still inherited by default
> - you can block inheritance explicitly by setting "__hash__ = None" in
> the class definition or on the class object
> - for a class defined in C, you can block __hash__ inheritance by
> setting the tp_hash slot to PyObject_HashNotImplemented
> - using one of the above approaches will cause hash(obj) to raise a
> TypeError, as will calling the tp_hash slot directly
> - unlike defining your own __hash__ method, using one of the above
> techniques will also mean that isinstance(obj, collections.Hashable)
> will also return False
> - defining __eq__ and/or __cmp__ without defining __hash__ will lead to
> a Py3k Warning being raised on the class definition when run with the -3
> switch
>
> The Python 3.0 behaviour is similar to the above, except that the
> unconditional inheritance of __hash__ is gone. If you define __eq__ or
> __cmp__ without defining __hash__, __hash__ will NOT be inherited from
> the parent class - instead, it will be set to None.
>
> The documentation of all of this could definitely use a bit more work -
> that's why I reopened 2235 in response to Michael's post.

Couple months ago I was trying to figure out how __eq__, __cmp__,
__hash__ and similar methods are called in Python and in which order
and wrote a docs here:

http://docs.sympy.org/python-comparisons.html

Feel free to reuse any of it. I wrote it to myself, as I didn't find
the official docs very clear.

Ondrej
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Documentation Error for __hash__

[Jesse Noller]

On Aug 30, 2008, at 2:41 PM, "Ondrej Certik" <[EMAIL PROTECTED]> wrote:

I strongly advise people interested in this topic to take a closer  
look

at the functionality that was added to address issue 2235. The "don't
inherit __hash__" behaviour has been backported from 3.0 and broke a
bunch of code, but the naive fix of reverting to the 2.5 behaviour
proceeded to make the 2.6 version of collections.Hashable completely
useless (since pretty much *everything* then claimed to be hashable,
including all of the container types in the standard library).

After thrashing out those problems, the 2.6 behaviour ended up being:
- __hash__ is still inherited by default
- you can block inheritance explicitly by setting "__hash__ = None"  
in

the class definition or on the class object
- for a class defined in C, you can block __hash__ inheritance by
setting the tp_hash slot to PyObject_HashNotImplemented
- using one of the above approaches will cause hash(obj) to raise a
TypeError, as will calling the tp_hash slot directly
- unlike defining your own __hash__ method, using one of the above
techniques will also mean that isinstance(obj, collections.Hashable)
will also return False
- defining __eq__ and/or __cmp__ without defining __hash__ will  
lead to
a Py3k Warning being raised on the class definition when run with  
the -3

switch

The Python 3.0 behaviour is similar to the above, except that the
unconditional inheritance of __hash__ is gone. If you define __eq__  
or
__cmp__ without defining __hash__, __hash__ will NOT be inherited  
from

the parent class - instead, it will be set to None.

The documentation of all of this could definitely use a bit more  
work -

that's why I reopened 2235 in response to Michael's post.


Couple months ago I was trying to figure out how __eq__, __cmp__,
__hash__ and similar methods are called in Python and in which order
and wrote a docs here:

http://docs.sympy.org/python-comparisons.html

Feel free to reuse any of it. I wrote it to myself, as I didn't find
the official docs very clear.

Ondrej


Ondrej, a patch that improves the official docs would be welcome and  
still potentially make 2.6/3.0

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Documentation Error for __hash__

[Ondrej Certik]

>> Ondrej
>
> Ondrej, a patch that improves the official docs would be welcome and still
> potentially make 2.6/3.0

That'd be awesome. I need to finish my thesis in the next couple days,
so I'd welcome if anyone could just take it and put usefult things in.
I could get to do it the next week the earliest.

Ondrej
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] The recursion checking problem

[Antoine Pitrou]

Hi,

I was working on a recursion overflow checking bug
(http://bugs.python.org/issue2548) and, while I've managed to produce a working
patch, I've also become uncomfortable with the very idea of trying to plug all
those holes just for the sake of plugging them. I'll try to explain why, by
describing the conflicting factors I've identified:

- more and more, we are adding calls to Py_EnterRecursiveCall() and
Py_LeaveRecursiveCall() all over the interpreter, to avoid special/obscure
cases of undetected infinite recursion; this can probably be considered a good
thing, but:

- after a recursion error has been raised (technically a RuntimeError), usually
some code has to do cleanup after noticing the exception; this cleanup now can
very easily bump into the recursion limit again, due to the point mentioned
above (the funniest example of this is PyErr_ExceptionMatches, which makes a
call to PyObject_IsSubclass which itself increases the recursion count because
__subclasscheck__ can be recursively invoked...).

- to counter the latter problem, py3k has introduced a somewhat smarter
mechanism (which I've tracked down to a commit in the defunct p3yk branch by
Martin): when the recursion limit is exceeded, a special flag named
"overflowed" is set in the thread state structure which disables the primary
recursion check, so that cleanup code has a bit of room to increase the
recursion count a bit. A secondary recursion check exists (equal to the primary
one /plus/ 50) and, if it is reached, the interpreter aborts with a fatal error.
The "overflowed" flag is cleared when the recursion count drops below the
primary recursion limit /minus/ 50. Now it looks rather smart but:

- unfortunately, some functions inside the interpreter discard every exception
by design. The primary example is PyDict_GetItem(), which is certainly used
quite a lot :-)... When PyDict_GetItem() returns NULL, the caller can only
assume that the key isn't in the dict, it has no way to know that there was a
critical problem due to a recursion overflow.

I encountered the latter problem when trying to backport the py3k recursion
overflow algorithm to trunk. A fatal error suddenly appeared in test_cpickle,
and it turned out that the recursion count was exceeded in
PyObject_RichCompare(), the error was then cleared in PyDict_GetItem(), but the
"overflowed" flag was still set so that a subsequent recursion overflow would
trigger the secondary check and lead to the fatal error.

I guess that, if it doesn't happen in py3k, it's just by chance: the recursion
overflow is probably happening at another point where errors don't get
discarded. Indeed, the failure I got on trunk was manifesting itself when
running "regrtest.py test_cpickle" but not directly "test_cpickle.py"... which
shows how delicate the recursion mechanism has become.

My attempt to solve the latter problem while still backporting the py3k scheme
involves clearing the "overflowed" flag in PyErr_Clear(). This makes all tests
pass ok, but also means the "overflowed" flag loses a lot of its meaning...
since PyErr_Clear() is called in a lot of places (and, especially, in
PyDict_GetItem()).


Also, at this point I fear that the solution to the problem is becoming,
because of its complexity, perhaps worse than the problem itself. That's why
I'm bringing it here, to have your opinion.

(I also suggest that we stop trying to fix recursion checking bugs until the
stable release, so as to give us some time to do the Right Thing later - if
there is such a thing)

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The recursion checking problem

[Brett Cannon]

On Sat, Aug 30, 2008 at 1:06 PM, Antoine Pitrou <[EMAIL PROTECTED]> wrote:
>
> Hi,
>
> I was working on a recursion overflow checking bug
> (http://bugs.python.org/issue2548) and, while I've managed to produce a 
> working
> patch, I've also become uncomfortable with the very idea of trying to plug all
> those holes just for the sake of plugging them. I'll try to explain why, by
> describing the conflicting factors I've identified:
>
> - more and more, we are adding calls to Py_EnterRecursiveCall() and
> Py_LeaveRecursiveCall() all over the interpreter, to avoid special/obscure
> cases of undetected infinite recursion; this can probably be considered a good
> thing, but:
>
> - after a recursion error has been raised (technically a RuntimeError), 
> usually
> some code has to do cleanup after noticing the exception; this cleanup now can
> very easily bump into the recursion limit again, due to the point mentioned
> above (the funniest example of this is PyErr_ExceptionMatches, which makes a
> call to PyObject_IsSubclass which itself increases the recursion count because
> __subclasscheck__ can be recursively invoked...).
>
> - to counter the latter problem, py3k has introduced a somewhat smarter
> mechanism (which I've tracked down to a commit in the defunct p3yk branch by
> Martin): when the recursion limit is exceeded, a special flag named
> "overflowed" is set in the thread state structure which disables the primary
> recursion check, so that cleanup code has a bit of room to increase the
> recursion count a bit. A secondary recursion check exists (equal to the 
> primary
> one /plus/ 50) and, if it is reached, the interpreter aborts with a fatal 
> error.
> The "overflowed" flag is cleared when the recursion count drops below the
> primary recursion limit /minus/ 50. Now it looks rather smart but:
>
> - unfortunately, some functions inside the interpreter discard every exception
> by design. The primary example is PyDict_GetItem(), which is certainly used
> quite a lot :-)... When PyDict_GetItem() returns NULL, the caller can only
> assume that the key isn't in the dict, it has no way to know that there was a
> critical problem due to a recursion overflow.
>

As the comment says for PyDict_GetItem(), it's a relic from the days
when there was no way to call Python code when making the call. That
is no longer true (and is probably true for a lot of places where a
similar assumption is made).

> I encountered the latter problem when trying to backport the py3k recursion
> overflow algorithm to trunk. A fatal error suddenly appeared in test_cpickle,
> and it turned out that the recursion count was exceeded in
> PyObject_RichCompare(), the error was then cleared in PyDict_GetItem(), but 
> the
> "overflowed" flag was still set so that a subsequent recursion overflow would
> trigger the secondary check and lead to the fatal error.
>
> I guess that, if it doesn't happen in py3k, it's just by chance: the recursion
> overflow is probably happening at another point where errors don't get
> discarded. Indeed, the failure I got on trunk was manifesting itself when
> running "regrtest.py test_cpickle" but not directly "test_cpickle.py"... which
> shows how delicate the recursion mechanism has become.
>
> My attempt to solve the latter problem while still backporting the py3k scheme
> involves clearing the "overflowed" flag in PyErr_Clear(). This makes all tests
> pass ok, but also means the "overflowed" flag loses a lot of its meaning...
> since PyErr_Clear() is called in a lot of places (and, especially, in
> PyDict_GetItem()).
>
>
> Also, at this point I fear that the solution to the problem is becoming,
> because of its complexity, perhaps worse than the problem itself. That's why
> I'm bringing it here, to have your opinion.
>

Well, For Py3K at least we might need to consider going through the C
API and fixing it so that these incorrect assumptions that functions
like PyDict_GetItem() make are no longer made by introducing some new
functions that behave in a "better" way.

And for the recursion issue, I think it stems from corners that are
cut in the C API by us. We inline functions all over the place, assume
that Python's implementation underneath the hood is going to make
calls that stay in C, etc. But as time has gone on and we have added
flexibility to Python, more and more places have a chance to call
Python code and trigger issues.

> (I also suggest that we stop trying to fix recursion checking bugs until the
> stable release, so as to give us some time to do the Right Thing later - if
> there is such a thing)
>

I have no problem punting for now; there is no way I would be willing
to wager any amount of money that the recursion check covered all
cases. I have always viewed the check as a bonus sanity check, but not
something to heavily rely upon.

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubs

[Python-Dev] [optparse] I wonder if optparse is dead ? because...

[KLEIN Stéphane]

Hello,

I wonder if optparse is dead ? because :

* svn access is down (svn co svn://starship.python.net/optik/trunk optik)
( http://sourceforge.net/mailarchive/forum.php?
thread_name=466F541C.6010804%40users.sourceforge.net&forum_name=optik-
users )

* last release is out 2 years ago
( http://sourceforge.net/mailarchive/forum.php?
thread_name=20060723161651.GA2076%40cthulhu.gerg.ca&forum_name=optik-
users )

* Greg Ward mail ( gward at python.net ) and is home page (http://
www.gerg.ca/) are away

* I'm interested by this feature
http://sourceforge.net/tracker/index.php?
func=detail&aid=1819510&group_id=38019&atid=421097
but it hasn't comment since one year ago.

Otherwise optparse is a great module !

Thanks for your information,
Stephane

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Documentation Error for __hash__

[Georg Brandl]

Ondrej Certik schrieb:
>>> Ondrej
>>
>> Ondrej, a patch that improves the official docs would be welcome and still
>> potentially make 2.6/3.0
> 
> That'd be awesome. I need to finish my thesis in the next couple days,
> so I'd welcome if anyone could just take it and put usefult things in.
> I could get to do it the next week the earliest.

Doc patches are always accepted, and I intend to update the online 2.6 and
3.0 docs continually even when the finals are released, so there is no
particular deadline you have to make.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [optparse] I wonder if optparse is dead ? because...

[Georg Brandl]

KLEIN Stéphane schrieb:
> Hello,
> 
> I wonder if optparse is dead ? because :
> 
> * svn access is down (svn co svn://starship.python.net/optik/trunk optik)
> ( http://sourceforge.net/mailarchive/forum.php?
> thread_name=466F541C.6010804%40users.sourceforge.net&forum_name=optik-
> users )
> 
> * last release is out 2 years ago
> ( http://sourceforge.net/mailarchive/forum.php?
> thread_name=20060723161651.GA2076%40cthulhu.gerg.ca&forum_name=optik-
> users )
> 
> * Greg Ward mail ( gward at python.net ) and is home page (http://
> www.gerg.ca/) are away
> 
> * I'm interested by this feature
> http://sourceforge.net/tracker/index.php?
> func=detail&aid=1819510&group_id=38019&atid=421097
> but it hasn't comment since one year ago.
> 
> Otherwise optparse is a great module !

Well, the module itself certainly isn't dead since it's distributed
with Python. What might be dead is the standalone version, Optik.

However, we have so far delegated feature requests to Greg and
expected them to be included in Optik first before being merged
to the core. If Optik is no longer maintained, we can do this
directly in the core, with Python's release cycle.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [optparse] I wonder if optparse is dead ? because...

[Tarek Ziadé]

On Sat, Aug 30, 2008 at 10:30 PM, KLEIN Stéphane <[EMAIL PROTECTED]>wrote:

> Hello,
>
> I wonder if optparse is dead ? because :
>
> * svn access is down (svn co svn://starship.python.net/optik/trunk optik)
> ( http://sourceforge.net/mailarchive/forum.php?
> thread_name=466F541C.6010804%40users.sourceforge.net&forum_name=optik-
> users )
>
> * last release is out 2 years ago
> ( http://sourceforge.net/mailarchive/forum.php?
> thread_name=20060723161651.GA2076%40cthulhu.gerg.ca&forum_name=optik-
> users )
>

Hello,

It has been integrated in Python since then, afaik

http://svn.python.org/projects/python/trunk/Lib/optparse.py


>
> * Greg Ward mail ( gward at python.net ) and is home page (http://
> www.gerg.ca/) are away
>
> * I'm interested by this feature
> http://sourceforge.net/tracker/index.php?
> func=detail&aid=1819510&group_id=38019&atid=421097
> but it hasn't comment since one year ago.
>
> Otherwise optparse is a great module !
>
> Thanks for your information,
> Stephane
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com
>



-- 
Tarek Ziadé | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [optparse] I wonder if optparse is dead ? because...

[KLEIN Stéphane]

Le Sat, 30 Aug 2008 23:15:13 +0200, Tarek Ziadé a écrit :

> On Sat, Aug 30, 2008 at 10:30 PM, KLEIN Stéphane
> <[EMAIL PROTECTED]>wrote:
> 
>> Hello,
>>
>> I wonder if optparse is dead ? because :
>>
>> * svn access is down (svn co svn://starship.python.net/optik/trunk
>> optik) ( http://sourceforge.net/mailarchive/forum.php?
>> thread_name=466F541C.6010804%40users.sourceforge.net&forum_name=optik-

>> users )
>>
>> * last release is out 2 years ago
>> ( http://sourceforge.net/mailarchive/forum.php?
>> thread_name=20060723161651.GA2076%40cthulhu.gerg.ca&forum_name=optik-

>> users )
>>
>>
> Hello,
> 
> It has been integrated in Python since then, afaik
> 
> http://svn.python.org/projects/python/trunk/Lib/optparse.py
> 

Yes, I know but there are this lines in optparse.py source code :

"""
By Greg Ward <[EMAIL PROTECTED]>

Originally distributed as Optik; see http://optik.sourceforge.net/ .

If you have problems with this module, please do not file bugs,
patches, or feature requests with Python; instead, use Optik's
SourceForge project page:
  http://sourceforge.net/projects/optik

For support, use the [EMAIL PROTECTED] mailing list
(http://lists.sourceforge.net/lists/listinfo/optik-users).
"""

then if I read this, I need to send my request feature to Optik web site 
but it appear asleep.

Regards,
Stephane

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Documentation Error for __hash__

[Ondrej Certik]

Hi Georg!

On Sat, Aug 30, 2008 at 11:09 PM, Georg Brandl <[EMAIL PROTECTED]> wrote:
> Ondrej Certik schrieb:
 Ondrej
>>>
>>> Ondrej, a patch that improves the official docs would be welcome and still
>>> potentially make 2.6/3.0
>>
>> That'd be awesome. I need to finish my thesis in the next couple days,
>> so I'd welcome if anyone could just take it and put usefult things in.
>> I could get to do it the next week the earliest.
>
> Doc patches are always accepted, and I intend to update the online 2.6 and
> 3.0 docs continually even when the finals are released, so there is no
> particular deadline you have to make.

This is awesome, I didn't know you take care of the official docs too.

I'll send the doc patches when I get to it, unless someone else does
it before me.

Ondrej
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com