[Python-Dev] Remove METH_OLDARGS?

2006-05-28 Thread Georg Brandl 
In the process of reviewing and possibly extending getargs.c, I stumbled
over the "compatibility" flag supposedly used for METH_OLDARGS functions.
No code in the core uses this calling convention any more, and it has been
deprecated for quite a long time (since 2.2), so would it be appropriate to end
support for it in 2.5?

[G]

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove METH_OLDARGS?

2006-05-28 Thread Neal Norwitz 
On 5/28/06, Georg Brandl <[EMAIL PROTECTED]> wrote:
> In the process of reviewing and possibly extending getargs.c, I stumbled
> over the "compatibility" flag supposedly used for METH_OLDARGS functions.
> No code in the core uses this calling convention any more, and it has been
> deprecated for quite a long time (since 2.2), so would it be appropriate to 
> end
> support for it in 2.5?

There's still a ton used under Modules.  Also, if no flag is
specified, it will default to 0 (ie, METH_OLDARGS).  I wonder how many
third party modules use METH_OLDARGS directly or more likely
indirectly.

The most important modules to fix from grep (and memory):

Modules/_sre.c:/* FIXME: use METH_OLDARGS instead of 0 or fix to
use METH_VARARGS */
Modules/_sre.c:/*METH_OLDARGS is not in Python 1.5.2 */
Modules/_tkinter.c: {"call",   Tkapp_Call, METH_OLDARGS},
Modules/_tkinter.c: {"globalcall", Tkapp_GlobalCall,
METH_OLDARGS},Modules/_tkinter.c: {"merge",
Tkapp_Merge, METH_OLDARGS},

$ grep -c METH_OLDARGS */*.c | egrep -v :0
Modules/_sre.c:2
Modules/_tkinter.c:3
Modules/audioop.c:24
Modules/clmodule.c:19
Modules/flmodule.c:103
Modules/fmmodule.c:6
Modules/glmodule.c:430
Modules/svmodule.c:34
Objects/methodobject.c:2

I would like to get rid of the flag, but I'm not sure we can do it
safely until 3.0.

n
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove METH_OLDARGS?

2006-05-28 Thread Raymond Hettinger 
> In the process of reviewing and possibly extending getargs.c, I stumbled
> over the "compatibility" flag supposedly used for METH_OLDARGS functions.
> No code in the core uses this calling convention any more, and it has been
> deprecated for quite a long time (since 2.2), so would it be appropriate to 
> end
> support for it in 2.5?

FWIW, I wouldn't miss it, nor would any of the code I've ever seen.


Raymond 
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal for a new itertools function: iwindow

2006-05-28 Thread Nick Coghlan 
Raymond Hettinger wrote:
> No thanks.  The resolution of this one was that windowing iterables is 
> not a good idea.  It is the natural province of sequences, not 
> iterables.  With sequences, it is a matter of using an index and 
> offset.  With iterables, there is a great deal of data shifting.  Also 
> note that some of the uses are subsumed by collections.deque().

Interesting - the old 'hammer-nail' tunnel vision strikes again, I guess.

So moving the question to a different part of the docs, would it make sense to 
include a deque recipe showing how to use a deque in a generator to permit 
windowing of an arbitrary iterator? Something like:

def window(iterable, window_len=2, window_step=1):
  itr = iter(iterable)
  step_range = xrange(window_step)
  current_window = collections.deque(islice(itr, window_len))
  while window_len == len(current_window):
  yield current_window
  for idx in step_range:
  current_window.popleft()
  current_window.extend(islice(itr, window_step))

Even if an application doesn't use the generator approach directly, it still 
illustrates how to use a deque for windowing instead of as a FIFO queue or a 
stack.

> The thought process was documented in a series of newsgroup postings:
>http://groups.google.com/group/comp.lang.python/msg/026da8f9eec4becf

Thanks for that reference - I was searching the wrong list, so I didn't find it.

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
 http://www.boredomandlaziness.org
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] Python Regression Test Failures refleak (101)

2006-05-28 Thread Tim Peters 
[... a huge number of reference leaks reported ...]

FYI, I "reduced" the relatively simple test_bisect's leaks to this
self-contained program:

libreftest = """
No actual doctests here.
"""

import doctest
import gc

def main():
from sys import gettotalrefcount as trc
for i in range(10):
doctest.testmod()
print trc()
doctest.master = None
gc.collect()

if __name__ == "__main__":
main()

Running that here in a debug build:

C:\Code\python\PCbuild>python_d blah.py
54867
54873
54879
54885
54891
54897
54903
54909
54915
54921

So it leaks 6 references per iteration, and merely invoking
doctest.testmod() is all it takes to provoke it.  Comment out the:

 test_support.run_doctest(test_bisect, verbose)

line in test_bisect.py, and test_bisect stops leaking too (which isn't
a trivial stmt:  its doctests are a small part of test_bisect --
_most_ of it isn't leaking).

What happens next isn't obvious to me -- nobody has touched doctest.py
in over 2 weeks, so that's not the _source_ of the problem.  Alas, I
won't have computer access for the next 13 hours or so, and have to
leave this here for now.

FWIW, the biggest change that went in since that last "normal" refleak
run is the new exception reworking, so that has to be a top suspect.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] test_gzip/test_tarfile failure om AMD64

2006-05-28 Thread Thomas Wouters 
I'm seeing a dubious failure of test_gzip and test_tarfile on my AMD64 machine. It's triggered by the recent struct changes, but I'd say it's probably caused by a bug/misfeature in zlibmodule: zlib.crc32 is the result of a zlib 'crc32' functioncall, which returns an unsigned long. 
zlib.crc32 turns that unsigned long into a (signed) Python int, which means a number beyond 1<<31 goes negative on 32-bit systems and other systems with 32-bit longs, but stays positive on systems with 64-bit longs:
(32-bit)>>> zlib.crc32("foobabazr")-271938108(64-bit)>>> zlib.crc32("foobabazr")4023029188The old structmodule coped with that:>>> 
struct.pack("'\xc4\x8d\xca\xef'>>> struct.pack("'\xc4\x8d\xca\xef'The new one does not:>>> struct.pack("'\xc4\x8d\xca\xef'>>> struct.pack("Traceback (most recent call last):  File "", line 1, in   File "Lib/struct.py", line 63, in pack
    return o.pack(*args)struct.error: 'l' format requires -2147483647 <= number <= 2147483647The structmodule should be fixed (and a test added ;) but I'm also wondering if zlib shouldn't be fixed. Now, I'm AMD64-centric, so my suggested fix would be to change the PyInt_FromLong() call to PyLong_FromUnsignedLong(), making zlib always return positive numbers -- it might break some code on 32-bit platforms, but that code is already broken on 64-bit platforms. But I guess I'm okay with the long being changed into an actual 32-bit signed number on 64-bit platforms, too.
-- Thomas Wouters <[EMAIL PROTECTED]>Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] Python Regression Test Failures refleak (101)

2006-05-28 Thread Thomas Wouters 
On 5/28/06, Tim Peters <[EMAIL PROTECTED]> wrote:
[... a huge number of reference leaks reported ...]FYI, I "reduced" the relatively simple test_bisect's leaks to thisself-contained program:Funny, I reduced it to more or less the same thing, except the other way 'round: I suspected exceptions to be the source of the leak, so I kept adding more complicated exception processing to my leak-test until it started leaking; it started leaking the moment I added a doctest to it, but not before. I somewhat doubt it's directly related to exceptions, though.
Michael Hudson also mentioned it on #nfs, and it seems he went a little further:00:54  i found that test.test_support.check_syntax reliably leaks 5    references00:55  as does compile('1=1', '', 'exec') in fact
00:58  it's leaking a tuple containing two Nones a string and an int, i    think01:00  oh well, i guess sean and richard know what they changed...Does 'a tuple containing two Nones, a string and an int' ring a bell to anyone? :)
-- Thomas Wouters <[EMAIL PROTECTED]>Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Iterating generator from C (PostgreSQL's pl/python RETUN SETOF/RECORD iterator support broken on RedHat buggy libs)

2006-05-28 Thread Thomas Wouters 
On 5/20/06, Hannu Krosing <[EMAIL PROTECTED]> wrote:
I try to move this to -dev as I hope there more people reading it whoare competent in internal working :). So please replay to -dev only.I'm not sure if you have found the problem on another mailinglist then, but I saw no answers on python-dev.
-The question is about use of generators in embedde v2.4 with asserts
enabled.Can somebody explain, why the code in try2.c works with wrappers 2 and 3but crashes on buggy exception for all others, that is pure generatorand wrappers 1,4,5 ?Your example code does not crash for me, not for any of the 'wrapper_source' variants, linking against Python 
2.4 (debian's python 2.4.3 on amd64, both in 64-bit and 32-bit mode) or Python 2.5 (current trunk, same hardware.) I don't know what kind of crash you were expecting, but I do see unchecked return values, which can cause crashes for the simplest of reasons ;-)
-- Thomas Wouters <[EMAIL PROTECTED]>Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] Python Regression Test Failures refleak (101)

2006-05-28 Thread Michael Hudson 
"Thomas Wouters" <[EMAIL PROTECTED]> writes:

> Does 'a tuple containing two Nones, a string and an int' ring a bell to
> anyone? :)

I found this one on the train (look at SyntaxError_init, it's
obvious).  I also found a number of other bugs in the new exceptions.c
code, from leaks:

>>> def f():
... try: UnicodeEncodeError()
... except TypeError: pass
... 
>>> for i in range(10):
... f()
... print sys.gettotalrefcount()
... 
30816
30821
30826
30831
30836
30841
30846
30851
30856
30861

to potential crashes:

>>> s = SystemExit(); s.code = []; s.__init__(); s.code
[[...]]
Bus error

So I think I'll be reading through exceptions.c pretty carefully.  I
don't think Sean and Richard have acquired as much paranoid
anal-mindedness and I have when hacking on Python C internals yet :)

Cheers,
mwh

-- 
  > Why are we talking about bricks and concrete in a lisp newsgroup?
  After long experiment it was found preferable to talking about why
  Lisp is slower than C++...
-- Duane Rettig & Tim Bradshaw, comp.lang.lisp
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 42 - New kind of Temporary file

2006-05-28 Thread Andrew Chambers 
The mailing-list bot said introduce myself so...

I'm Andy Chambers.  I've been lurking on the web-interface to the list
for a while.  I recently started trying to implement one of the items on
PEP 42 so I thought I should join the list and make myself known.

The item I'm working on is the new kind of temporary file that only
turns into a file when required (before which time presumably its a
StringIO).

Do we still want this?  I suggest it be exposed as a class TempFile in
the tempfile module.  If yes, I'll write the test cases and submit a
patch.

Regards,
Andy



___ 
Inbox full of spam? Get leading spam protection and 1GB storage with All New 
Yahoo! Mail. http://uk.docs.yahoo.com/nowyoucan.html
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] Python Regression Test Failures refleak (101)

2006-05-28 Thread Georg Brandl 
Michael Hudson wrote:

> So I think I'll be reading through exceptions.c pretty carefully.  I
> don't think Sean and Richard have acquired as much paranoid
> anal-mindedness and I have when hacking on Python C internals yet :)

I intended to go through the code again today or tomorrow, looking for
things we missed, but I was on the plane until now, so obviously you
were faster.

Thanks for doing our work, though ;)

Georg

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove METH_OLDARGS?

2006-05-28 Thread Georg Brandl 
Neal Norwitz wrote:
> On 5/28/06, Georg Brandl <[EMAIL PROTECTED]> wrote:
>> In the process of reviewing and possibly extending getargs.c, I stumbled
>> over the "compatibility" flag supposedly used for METH_OLDARGS functions.
>> No code in the core uses this calling convention any more, and it has been
>> deprecated for quite a long time (since 2.2), so would it be appropriate to 
>> end
>> support for it in 2.5?
> 
> There's still a ton used under Modules.  Also, if no flag is
> specified, it will default to 0 (ie, METH_OLDARGS).  I wonder how many
> third party modules use METH_OLDARGS directly or more likely
> indirectly.

These modules can be converted.

> I would like to get rid of the flag, but I'm not sure we can do it
> safely until 3.0.

It has been deprecated since 2.2, so it'd be no surprise if it went away. We'd
still have to support PyArg_Parse since it's a public API function and not
deprecated, but it could just convert the arguments to new style and call
PyArg_ParseTuple.

Also, it would be easy to detect METH_OLDARGS in PyCFunction_New and raise
an appropriate exception.

It's not my decision though.

Georg

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] Python Regression Test Failures refleak (101)

2006-05-28 Thread Michael Hudson 
Georg Brandl <[EMAIL PROTECTED]> writes:

> Michael Hudson wrote:
>
>> So I think I'll be reading through exceptions.c pretty carefully.  I
>> don't think Sean and Richard have acquired as much paranoid
>> anal-mindedness and I have when hacking on Python C internals yet :)
>
> I intended to go through the code again today or tomorrow, looking for
> things we missed, but I was on the plane until now, so obviously you
> were faster.

Oh right.  I'd have preferred you didn't check in unfinished
code to the trunk, I think...

> Thanks for doing our work, though ;)

It has to be said, the degree of the refleaks put me into emergency
fire fighting mode...

Cheers,
mwh

-- 
  You have run into the classic Dmachine problem: your machine has
  become occupied by a malevolent spirit.  Replacing hardware or
  software will not fix this - you need an exorcist. 
   -- Tim Bradshaw, comp.lang.lisp
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] Python Regression Test Failures refleak (101)

2006-05-28 Thread Georg Brandl 
Michael Hudson wrote:
> Georg Brandl <[EMAIL PROTECTED]> writes:
> 
>> Michael Hudson wrote:
>>
>>> So I think I'll be reading through exceptions.c pretty carefully.  I
>>> don't think Sean and Richard have acquired as much paranoid
>>> anal-mindedness and I have when hacking on Python C internals yet :)
>>
>> I intended to go through the code again today or tomorrow, looking for
>> things we missed, but I was on the plane until now, so obviously you
>> were faster.
> 
> Oh right.  I'd have preferred you didn't check in unfinished
> code to the trunk, I think...

I blame it on Tim since he said OK ;)

>> Thanks for doing our work, though ;)
> 
> It has to be said, the degree of the refleaks put me into emergency
> fire fighting mode...

I hadn't expected so many of them, but I also have to admit that we didn't have
that much time on Saturday and wanted the branch merged for obvious reasons...

[G]

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] Python Regression Test Failures refleak (101)

2006-05-28 Thread Michael Hudson 
Georg Brandl <[EMAIL PROTECTED]> writes:

> Michael Hudson wrote:
>> Georg Brandl <[EMAIL PROTECTED]> writes:
>> 
>>> Michael Hudson wrote:
>>>
 So I think I'll be reading through exceptions.c pretty carefully.  I
 don't think Sean and Richard have acquired as much paranoid
 anal-mindedness and I have when hacking on Python C internals yet :)
>>>
>>> I intended to go through the code again today or tomorrow, looking for
>>> things we missed, but I was on the plane until now, so obviously you
>>> were faster.
>> 
>> Oh right.  I'd have preferred you didn't check in unfinished
>> code to the trunk, I think...
>
> I blame it on Tim since he said OK ;)

I hope he feels bad about it too :)

>>> Thanks for doing our work, though ;)
>> 
>> It has to be said, the degree of the refleaks put me into emergency
>> fire fighting mode...
>
> I hadn't expected so many of them, but I also have to admit that we didn't 
> have
> that much time on Saturday and wanted the branch merged for obvious reasons...

You can make it up to me by writing unit tests for the crash cases
I've fixed.

I think I've fixed the refleaks too, but running regrtest -R :: takes
rther a while.

Cheers,
mwh

-- 
  QNX... the OS that walks like a duck, quacks like a duck, but is,
  in fact, a platypus. ... the adventures of porting duck software 
  to the platypus were avoidable this time.
 -- Chris Klein, alt.sysadmin.recovery
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] DRAFT: python-dev summary for 2006-03-16 to 2006-03-31

2006-05-28 Thread Steven Bethard 
Sorry I'm so far behind -- I'm aiming to get mostly caught up this
week.  Please read through the summary if you have the time and send
me any comments or corrections.

Thanks!


=
Announcements
=

---
Python 2.5 schedule
---

Python 2.5 is moving forward along its release schedule.  A few issues
still remain; check `PEP 356`_ for details.

.. _PEP 356: http://www.python.org/dev/peps/pep-0356/

Contributing thread:

- `Python 2.5 Schedule
`__

-
Python 3000 gets its own list
-

Serious work on Python 3000 has started now, with a new `python-3000
mailing list`_ for general discussion, and a `python-3000-checkins
mailing list`_ for checkins to the p3yk branch.  Right now, Guido
wants patches to focus mainly on ripping out old code (e.g. classic
classes) and wants the discussion to focus primarily on formalizing
the processes for introducing Python 3000.

Note that processes is what's important right now

.. _python-3000 mailing list:
http://mail.python.org/mailman/listinfo/python-3000
.. _python-3000-checkins mailing list:
http://mail.python.org/mailman/listinfo/python-3000-checkins

Contributing threads:

- `Python 3000 Process
`__
- `svn checkins are now split
`__
- `Discussing the Great Library Reorganization
`__

---
Buildbot warnings redirected to python-checkins
---

The buildbot warnings, which for a short time this month appeared on
python-dev, have been redirected to the python-checkins list.

Contributing thread:

- `[Fwd: buildbot warnings in amd64 gentoo trunk]
`__

--
Checkins that change behavior must be accompanied by tests
--

Though it's always been good practice to check in a test with any
behavior-changing patch, Neal Norwitz has requested extra care in this
area as we approach the release of Python 2.5.  From this point on,
*any* checkin that could change behavior should be accompanied by a
corresponding test.

Contributing thread:

- `improving quality
`__

--
Email 4.0 merged into Python 2.5 trunk
--

Barry Warsaw has merged the Email 4.0 package into the Python 2.5
trunk. There's a minor incompatibility in that email.Parser is now a
placeholder object instead of a module, but this should not affect the
vast majority of users.

Contributing thread:

- `Merging email 4.0 to Python 2.5 svn trunk
`__

-
Python 2.4.3 released
-

`Python 2.4.3`_ was released on March 30th. This fixes over 50 bugs in
Python 2.4.2, so now's the time to upgrade.

.. _Python 2.4.3: http://www.python.org/2.4.3/

Contributing threads:

- `release24-maint FREEZE for 2.4.3c1, this Thursday
`__
- `RELEASED Python 2.4.3, release candidate 1
`__
- `release24-maint unfrozen, kinda
`__
- `BRANCH release24-maint FREEZE from 00:00 UTC on Wednesday 29th
`__
- `RELEASED Python 2.4.3, final.
`__


=
Summaries
=


Including pysqlite in the stdlib


Anthony Baxter asked if folks were still interested in including
pysqlite_ in Python 2.5 and got a pretty positive response. Gerhard
Häring said he could release pysqlite 2.2 and then merge that into
Python SVN trunk.  In the future, he agreed to synchronize stable
changes in the standalone release with the Python core sqlite module.
Accompanying these agreements was a long discussion about the pros and
cons of including pysqlite, and of course an endless debate about what
the new module should be named.  In the end, Guido approved the
inclusion of pysqlite, and it was given the name sqlite3.

.. _pysqlite: http://initd.org/tracker/pysqlite/

Contributing thread:

- `pysqlite for 2.5?
`__

-
Choosing a new tracker system
-

After Guido noticed that SourceForge had st

[Python-Dev] dictionary order

2006-05-28 Thread Armin Rigo 
Hi all,

I'm playing with dicts that mangle the hash they receive before using it
for hashing.  The goal was to detect obscure dict order dependencies in
my own programs, but I couldn't resist and ran the Python test suite
with various mangling schemes.  As expected -- what is not tested is
broken -- I found and fixed tons of small dependencies in the tests
themselves, plus one in base64.py.

Now I'm stumbling upon this test for urllib2:

>>> mgr = urllib2.HTTPPasswordMgr()
>>> add = mgr.add_password
>>> add("Some Realm", "http://example.com/";, "joe", "password")
>>> add("Some Realm", "http://example.com/ni";, "ni", "ni")
(...)

Currently, we use the highest-level path where more than one
match:

>>> mgr.find_user_password("Some Realm", "http://example.com/ni";)
('joe', 'password')

Returning the outermost path is a bit strange, if you ask me, but I am
no expert here.  Stranger is the fact that the actual implement actually
returns, not the outermost path at all -- there is no code to do that --
but a random pick, the first match in dictionary order.  The comment in
the test is just misleading.  I believe that urllib2 should be fixed to
always return the *innermost* path, but I need confirmation about
this...


A bientot,

Armin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] Python Regression Test Failures refleak (101)

2006-05-28 Thread Michael Hudson 
Michael Hudson <[EMAIL PROTECTED]> writes:

> I think I've fixed the refleaks too, but running regrtest -R :: takes
> rther a while.

I hadn't: test_codecs and test_codeccallbacks both leak, the latter
quite spectacularly (9000+ refleaks a run).  The test_codecs leaks
come from calls to codecs.decode_charmap, I'm not sure that it's to do
with the exceptions.c changes.  The test_codeccallbacks leaks seem
more like they could be exception related: when a unicode decoding
fails, two references are leaked.

I've run out of energy for these tonight.

Cheers,
mwh

-- 
  To summarise the summary of the summary:- people are a problem.
   -- The Hitch-Hikers Guide to the Galaxy, Episode 12
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] Python Regression Test Failures refleak (101)

2006-05-28 Thread Georg Brandl 
Michael Hudson wrote:
> Michael Hudson <[EMAIL PROTECTED]> writes:
> 
>> I think I've fixed the refleaks too, but running regrtest -R :: takes
>> rther a while.
> 
> I hadn't: test_codecs and test_codeccallbacks both leak, the latter
> quite spectacularly (9000+ refleaks a run).  The test_codecs leaks
> come from calls to codecs.decode_charmap, I'm not sure that it's to do
> with the exceptions.c changes.  The test_codeccallbacks leaks seem
> more like they could be exception related: when a unicode decoding
> fails, two references are leaked.
> 
> I've run out of energy for these tonight.

I repaired that, it was in the GetStart/GetEnd functions for UnicodeErrors.

Currently running -R ::, so let's see...

Georg

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove METH_OLDARGS?

2006-05-28 Thread Martin v. Löwis 
Georg Brandl wrote:
>> There's still a ton used under Modules.  Also, if no flag is
>> specified, it will default to 0 (ie, METH_OLDARGS).  I wonder how many
>> third party modules use METH_OLDARGS directly or more likely
>> indirectly.
> 
> These modules can be converted.

I think that should be done for 2.5, but nothing else. Or, perhaps
a deprecation warning could be generated if it is still used.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] dictionary order

2006-05-28 Thread John J Lee 
On Sun, 28 May 2006, Armin Rigo wrote:
[...]
> Now I'm stumbling upon this test for urllib2:
>
>>>> mgr = urllib2.HTTPPasswordMgr()
>>>> add = mgr.add_password
>>>> add("Some Realm", "http://example.com/";, "joe", "password")
>>>> add("Some Realm", "http://example.com/ni";, "ni", "ni")
>(...)
>
>Currently, we use the highest-level path where more than one
>match:
>
>>>> mgr.find_user_password("Some Realm", "http://example.com/ni";)
>('joe', 'password')
>
> Returning the outermost path is a bit strange, if you ask me, but I am
> no expert here.  Stranger is the fact that the actual implement actually
> returns, not the outermost path at all -- there is no code to do that --
> but a random pick, the first match in dictionary order.  The comment in
> the test is just misleading.  I believe that urllib2 should be fixed to
> always return the *innermost* path, but I need confirmation about
> this...

I noticed the same things, and in fact I think this was fixed before you 
posted :-)

FWIW, here are the details:

The checkin was from Georg as -r 46509, patch was 
http://python.org/sf/1496206

Part of the comment on the patch:

"""
The patch also comments out one test which was testing
something not actually guaranteed by the code at all --
it was passing by fluke. The code it's trying to test
could do with some review, which is why I left this
test commented out rather than deleting the test (but
that is a long-standing issue unrelated to this patch,
so should not block this patch from being applied).
"""

Recently I have been slowly working my way through urllib2 auth, fixing 
bugs and adding tests as I go.  This particular issue is horribly unclear 
in the RFC, though, and I haven't yet got round to the necessary checking 
of real-world behaviour.


John

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Syntax errors on continuation lines

2006-05-28 Thread Roger Miller 
I am a recent subscriber to this list.  A couple of months ago
I downloaded the Python source out of curiosity, then decided
to see if I could scratch a couple of small itches.

One of them was syntax errors reported on one line but actually
resulting from unbalanced brackets on the previous line. (The
folks on this list may not make such silly mistakes any more,
but several postings on comp.lang.python suggest that I am not
entirely alone in finding it an annoyance.)  I have modified
the parser and error reporting code to provide more explicit
information in such cases, and am now looking for some guidance
as to whether it is worth submitting a patch.

With my change a syntax error on a continuation line looks like
this:

  File "test.py", line 42 (statement started on line 41)
y = 0
  ^
SyntaxError: invalid syntax

The patch adds a new attribute to the SytaxError exception
object (the line number on which the statement started).  I
don't have the experience to judge whether this would be
considered a compatibility problem.  An alternative that does
not require modifying the exception would be to change the
message text without identifying the specific line number:

  File "test.py", line 42
y = 0
  ^
SyntaxError: invalid syntax in continuation of prior line

However I think the continuation indication properly belongs
with the location information rather than with the message.
(The loss of the starting line number itself is no big deal;
it will usually be one less than the error line number.)


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_gzip/test_tarfile failure om AMD64

2006-05-28 Thread Bob Ippolito 

On May 28, 2006, at 4:31 AM, Thomas Wouters wrote:

>
> I'm seeing a dubious failure of test_gzip and test_tarfile on my  
> AMD64 machine. It's triggered by the recent struct changes, but I'd  
> say it's probably caused by a bug/misfeature in zlibmodule:  
> zlib.crc32 is the result of a zlib 'crc32' functioncall, which  
> returns an unsigned long. zlib.crc32 turns that unsigned long into  
> a (signed) Python int, which means a number beyond 1<<31 goes  
> negative on 32-bit systems and other systems with 32-bit longs, but  
> stays positive on systems with 64-bit longs:
>
> (32-bit)
> >>> zlib.crc32("foobabazr")
> -271938108
>
> (64-bit)
> >>> zlib.crc32("foobabazr")
> 4023029188
>
> The old structmodule coped with that:
> >>> struct.pack(" '\xc4\x8d\xca\xef'
> >>> struct.pack(" '\xc4\x8d\xca\xef'
>
> The new one does not:
> >>> struct.pack(" '\xc4\x8d\xca\xef'
> >>> struct.pack(" Traceback (most recent call last):
>   File "", line 1, in 
>   File "Lib/struct.py", line 63, in pack
> return o.pack(*args)
> struct.error: 'l' format requires -2147483647 <= number <= 2147483647
>
> The structmodule should be fixed (and a test added ;) but I'm also  
> wondering if zlib shouldn't be fixed. Now, I'm AMD64-centric, so my  
> suggested fix would be to change the PyInt_FromLong() call to  
> PyLong_FromUnsignedLong(), making zlib always return positive  
> numbers -- it might break some code on 32-bit platforms, but that  
> code is already broken on 64-bit platforms. But I guess I'm okay  
> with the long being changed into an actual 32-bit signed number on  
> 64-bit platforms, too.

The struct module isn't what's broken here. All of the struct types  
have always had well defined bit sizes and alignment if you  
explicitly specify an endian, >I and >L are 32-bits everywhere, and  
 >Q is supported on platforms that don't have long long. The only  
thing that's changed is that it actually checks for errors  
consistently now.

-bob

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_gzip/test_tarfile failure om AMD64

2006-05-28 Thread Thomas Wouters 
On 5/29/06, Bob Ippolito <[EMAIL PROTECTED]> wrote:
On May 28, 2006, at 4:31 AM, Thomas Wouters wrote:>> I'm seeing a dubious failure of test_gzip and test_tarfile on my> AMD64 machine. It's triggered by the recent struct changes, but I'd> say it's probably caused by a bug/misfeature in zlibmodule:
> zlib.crc32 is the result of a zlib 'crc32' functioncall, which> returns an unsigned long. zlib.crc32 turns that unsigned long into> a (signed) Python int, which means a number beyond 1<<31 goes
> negative on 32-bit systems and other systems with 32-bit longs, but> stays positive on systems with 64-bit longs:>> (32-bit)> >>> zlib.crc32("foobabazr")> -271938108
>> (64-bit)> >>> zlib.crc32("foobabazr")> 4023029188>> The old structmodule coped with that:> >>> struct.pack("
> '\xc4\x8d\xca\xef'> >>> struct.pack("> '\xc4\x8d\xca\xef'>> The new one does not:> >>> struct.pack("> '\xc4\x8d\xca\xef'
> >>> struct.pack("> Traceback (most recent call last):>   File "", line 1, in >   File "Lib/struct.py", line 63, in pack
> return o.pack(*args)> struct.error: 'l' format requires -2147483647 <= number <= 2147483647>> The structmodule should be fixed (and a test added ;) but I'm also> wondering if zlib shouldn't be fixed. Now, I'm AMD64-centric, so my
> suggested fix would be to change the PyInt_FromLong() call to> PyLong_FromUnsignedLong(), making zlib always return positive> numbers -- it might break some code on 32-bit platforms, but that> code is already broken on 64-bit platforms. But I guess I'm okay
> with the long being changed into an actual 32-bit signed number on> 64-bit platforms, too.The struct module isn't what's broken here. All of the struct typeshave always had well defined bit sizes and alignment if you
explicitly specify an endian, >I and >L are 32-bits everywhere, and >Q is supported on platforms that don't have long long. The onlything that's changed is that it actually checks for errorsconsistently now.
Yes. And that breaks things. I'm certain the behaviour is used in real-world code (and I don't mean just the gzip module.) It has always, as far as I can remember, accepted 'unsigned' values for the signed versions of ints, longs and long-longs (but not chars or shorts.) I agree that that's wrong, but I don't think changing struct to do the right thing should be done in 
2.5. I don't even think it should be done in 2.6 -- although 3.0 is fine.-- Thomas Wouters <[EMAIL PROTECTED]>Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] Python Regression Test Failures refleak (101)

2006-05-28 Thread Neal Norwitz 
On 5/28/06, Michael Hudson <[EMAIL PROTECTED]> wrote:
>
> I think I've fixed the refleaks too, but running regrtest -R :: takes
> rther a while.

FYI, I typically run -R 4:3: since it seems to do a pretty good job
and takes 20% less time.  I never run -R with -u all, only a normal
run or specific tests.  (-u all and -R can be done by the automated
run.)

n
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove METH_OLDARGS?

2006-05-28 Thread Neal Norwitz 
On 5/28/06, Georg Brandl <[EMAIL PROTECTED]> wrote:
> Neal Norwitz wrote:
> > On 5/28/06, Georg Brandl <[EMAIL PROTECTED]> wrote:
> >> In the process of reviewing and possibly extending getargs.c, I stumbled
> >> over the "compatibility" flag supposedly used for METH_OLDARGS functions.
> >> No code in the core uses this calling convention any more, and it has been
> >> deprecated for quite a long time (since 2.2), so would it be appropriate 
> >> to end
> >> support for it in 2.5?
> >
> > There's still a ton used under Modules.  Also, if no flag is
> > specified, it will default to 0 (ie, METH_OLDARGS).  I wonder how many
> > third party modules use METH_OLDARGS directly or more likely
> > indirectly.
>
> These modules can be converted.

We can convert the one's in Python, but not the third party ones.
It's also very difficult to search for something that's not there.
You need to review every PyMethodDef entry to verify it has a METH_*
flag.  These implicit METH_OLDARGS may well exist in Python,
especially given who did the patch to add them in the first place. :-)

http://mail.python.org/pipermail/python-dev/2002-March/022009.html
http://mail.python.org/pipermail/patches/2002-July/009023.html

> > I would like to get rid of the flag, but I'm not sure we can do it
> > safely until 3.0.
>
> It has been deprecated since 2.2, so it'd be no surprise if it went away.

There's a difference between "it'd be no surprise" and "it should be
no surpise".  While I agree in theory they are the same, in practice,
they are not.  That's the difference between theory and practice. :-)

> Also, it would be easy to detect METH_OLDARGS in PyCFunction_New and raise
> an appropriate exception.

I agree with Martin this should raise a deprecation warning in 2.5.

n
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] ssize_t question: longs in header files

2006-05-28 Thread Neal Norwitz 

$ grep long */*.h

minus comments, etc  yields several questions about whether some
values should use Py_ssize_t rather than C longs.  In particular:

* hash values
Include/abstract.h: long PyObject_Hash(PyObject *o);  // also in object.h
Include/object.h:typedef long (*hashfunc)(PyObject *);

* ints:  Include/intobject.h:long ob_ival;
* thread values (probably no need to change since platform dependent)

Attached is the whole list.  This is just for C longs.  The int list
is much bigger and coming in a different msg.

n


header-longs
Description: Binary data
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] ssize_t: ints in header files

2006-05-28 Thread Neal Norwitz 
Here's the int questions.  The entire list is too big to post (50k),
so it's here:

http://www.python.org/neal/header-ints

Should the following values be ints (limited to 2G)?

 * dict counts (ma_fill, ma_used, ma_mask)
 * line #s and column #s
 * AST (asdl.h) sequences
 * recursion limit
 * read/write/send/recv results and buf size
 * code object values like # args, # locals, stacksize, # instructions
 * sre (i think i have a patch to fix this somewhere)

Please try to review the APIs and let us know if there are any values
you think should be 64-bits on 64-bit platforms.

n
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com