Re: [Python-Dev] need info for externally maintained modules PEP
Brett Cannon wrote: > - expat Not sure whether you mean the Expat parser proper here, or the pyexpat module: both are externally maintained, also; pyexpat is part of PyXML. OTOH, people can just consider PyXML to be part of Python, and I synchronize the Python sources with the PyXML sources from time to time. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] need info for externally maintained modules PEP
On 4/9/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Brett Cannon wrote: > > - expat > > Not sure whether you mean the Expat parser proper here, or the pyexpat > module: both are externally maintained, I was thinking the parser, but if pyexpat is externally maintained, then both of them. > also; pyexpat is part of PyXML. > OTOH, people can just consider PyXML to be part of Python, and I > synchronize the Python sources with the PyXML sources from time to time. OK, so then just the info for expat. -Brett ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Who understands _ssl.c on Windows?
Tim Peters wrote: > Anyone: is it "a bug" or "a feature" that a socket.connect() call > that times out may raise either socket.timeout or socket.error? > Trent's box raises socket.error. On my box, when I set a timeout > value small enough so that it _does_ time out, socket.timeout is > raised: That should give another clue as to what happened: It is *not* the select call that timed out - this would have raised a socket_timeout exception, due to internal_connect setting the timeout variable. Looking at the code, the following operations might have caused this error 10060: - getsockaddrarg - connect, returning WSAETIMEDOUT instead of WSAEWOULDBLOCK The code in getsockaddrarg is hard to follow, but I think it should only ever cause socket.gaierror. So why would connect ignore the non-blocking mode? the earlier call to ioctlsocket might have failed - unfortunately, we wouldn't know about that, because the return value of ioctlsocket (and all other system calls in internal_setblocking) is ignored. You might want to take a look at this KB article: http://support.microsoft.com/kb/179942/EN-US/ which claims that the WSA_FLAG_OVERLAPPED flag must be set on a socket on NT 4.0 for non-blocking mode to work. This shouldn't matter here, as the system is W2k, and because we used the socket API to create the socket (right?). Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Subversion downtime today
I will need to turn off subversion write access for an hour or so today, from about 18:00 GMT to 19:00 GMT. During that time, I will load the stackless svn dumpfile into the repository. I don't expect problems, but if there are any, I want to be able to revert to a backup, without losing any unrelated commits. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Win64 AMD64 (aka x64) binaries available64
Thomas Heller wrote:
>>> return Py_BuildValue("s",
>>> ver.dwMajorVersion,
>>> ver.dwMinorVersion,
>>> ver.dwBuildNumber,
>>> ver.dwPlatformId,
>>> ver.szCSDVersion);
>>>
>>> The crash disappears if I change the first parameter in the
>>> Py_BuildValue call to "s". No idea why.
>>> With this change, I can start the exe without a crash, but
>>> sys.versioninfo starts with (IIRC) (2, 0, 5,...).
>> Very strange. What is your compiler version (first line of cl /?)?
I have looked into this. In the latest SDK (2003 SP1), Microsoft has
changed the include structure; there are no separate amd64
subdirectories anymore. Then, cl.exe was picking up the wrong
stdarg.h (the one of VS 2003), which would not work for AMD64.
I have corrected that in vsextcomp, but I will need to check a few
more things before releasing it.
Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] int()'s ValueError behaviour
Someone on IRC (who refuses to report bugs on sourceforge, so I guess he wants to remain anonymous) came with this very amusing bug: int(), when raising ValueError, doesn't quote (or repr(), rather) its arguments:
>>> int("")Traceback (most recent call last): File "", line 1, in ?ValueError: invalid literal for int():>>> int("34\n\n\n5")Traceback (most recent call last):
File "", line 1, in ?ValueError: invalid literal for int(): 345>>>Unicode behaviour also isn't always consistent:>>> int(u'\u0100')Traceback (most recent call last):
File "", line 1, in ?UnicodeEncodeError: 'decimal' codec can't encode character u'\u0100' in position 0: invalid decimal Unicode string>>> int(u'\u09ec', 6)Traceback (most recent call last):
File "", line 1, in ?ValueError: invalid literal for int(): 6And trying to use the 'decimal' codec directly:>>> u'6'.encode('decimal')Traceback (most recent call last):
File "", line 1, in ?LookupError: unknown encoding: decimalI'm not sure if the latter problems are fixable, but the former should be fixed by passing the argument to ValueError through repr(), I think. It's also been suggested (by the reporter, and I agree) that the actual base should be in the errormessage too. Is there some reason not to do this that I've overlooked?
-- Thomas Wouters <[EMAIL PROTECTED]>Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] need info for externally maintained modules PEP
On 4/8/06, Brett Cannon <[EMAIL PROTECTED]> wrote: > Anyway, here is a list of the packages that I think have outside > maintenance (or at least have been at some point). Anyone who has > info on them that I need, please let me know the details. Also, if I > missed any, obviously speak up: I think there's still some confusion over optparse/optik. There was a recent thread about a feature in optik which hadn't made it into optparse - but maybe that's just a synchronisation problem. Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] need info for externally maintained modules PEP
At 02:47 PM 4/8/2006 -0700, Brett Cannon wrote: > Do people think we need to document the version that has >been imported into Python and when that was done as well? Definitely. Better yet, in 2.5 I'd like to have the version information embedded in a PKG-INFO file that gets installed with Python, so that setuptools can detect the package's presence. (This is particularly important for cytpes, sqlite, and ElementTree, which were added in 2.5 and thus may be listed in package dependencies for 2.3/2.4 programs.) Unfortunately, I haven't quite figured out how to ensure that the PKG-INFO files get built, because there's only one setup.py in the Python build process at the moment. It would be good if we could have separate setup.py files for "external" libraries, not only because of the ability to get PKG-INFO installed, but also because then OS vendors that split those externals out into separate system packages wouldn't need to jump through as many hoops to build just a single external library like ElementTree or ctypes. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] int()'s ValueError behaviour
Go ahead and fix it. This was probably never changed since 1990 or
so... Do expect some code brakage where people rely on the old
behavior. :-(
--Guido
On 4/9/06, Thomas Wouters <[EMAIL PROTECTED]> wrote:
>
> Someone on IRC (who refuses to report bugs on sourceforge, so I guess he
> wants to remain anonymous) came with this very amusing bug: int(), when
> raising ValueError, doesn't quote (or repr(), rather) its arguments:
>
> >>> int("")
> Traceback (most recent call last):
> File "", line 1, in ?
> ValueError: invalid literal for int():
> >>> int("34\n\n\n5")
> Traceback (most recent call last):
> File "", line 1, in ?
> ValueError: invalid literal for int(): 34
>
>
> 5
> >>>
>
> Unicode behaviour also isn't always consistent:
> >>> int(u'\u0100')
> Traceback (most recent call last):
> File "", line 1, in ?
> UnicodeEncodeError: 'decimal' codec can't encode character u'\u0100' in
> position 0: invalid decimal Unicode string
> >>> int(u'\u09ec', 6)
> Traceback (most recent call last):
> File "", line 1, in ?
> ValueError: invalid literal for int(): 6
>
> And trying to use the 'decimal' codec directly:
> >>> u'6'.encode('decimal')
> Traceback (most recent call last):
> File "", line 1, in ?
> LookupError: unknown encoding: decimal
>
> I'm not sure if the latter problems are fixable, but the former should be
> fixed by passing the argument to ValueError through repr(), I think. It's
> also been suggested (by the reporter, and I agree) that the actual base
> should be in the errormessage too. Is there some reason not to do this that
> I've overlooked?
>
> --
> Thomas Wouters <[EMAIL PROTECTED]>
>
> Hi! I'm a .signature virus! copy me into your .signature file to help me
> spread!
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>
>
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] threadless brownian.py
Recently I downloaded Python 2.5a1 and I have started playing with it. In doing so, I have been looking at the Demo directory in the distribution, to check if demos of the new features have been added there. So, I rediscovered brownian.py, in Demo/tkinter/guido. I just love this little program, because it reminds myself of one my first programs, a long long time ago (a brownian motion in AmigaBasic, with sprites!). It is also one of the first programs I looked at, when I started studying threads four years ago and I thought it was perfect. However, nowadays I know better and I have realized that brownian.py is perfect textbook example of a case where you don't really need threads, and you can use generators instead. So I thought it would be nice to add a threadless version of brownian.py in the Demo directory. Here it is. If you like it, I donate the code to the PSF! # Brownian motion -- an example of a NON multi-threaded Tkinter program ;) from Tkinter import * import random import sys WIDTH = 400 HEIGHT = 300 SIGMA = 10 BUZZ = 2 RADIUS = 2 LAMBDA = 10 FILL = 'red' stop = 0# Set when main loop exits root = None # main window def particle(canvas): # particle = iterator over the moves r = RADIUS x = random.gauss(WIDTH/2.0, SIGMA) y = random.gauss(HEIGHT/2.0, SIGMA) p = canvas.create_oval(x-r, y-r, x+r, y+r, fill=FILL) while not stop: dx = random.gauss(0, BUZZ) dy = random.gauss(0, BUZZ) try: canvas.move(p, dx, dy) except TclError: break else: yield None def move(particle): # move the particle at random time particle.next() dt = random.expovariate(LAMBDA) root.after(int(dt*1000), move, particle) def main(): global root, stop root = Tk() canvas = Canvas(root, width=WIDTH, height=HEIGHT) canvas.pack(fill='both', expand=1) np = 30 if sys.argv[1:]: np = int(sys.argv[1]) for i in range(np): # start the dance move(particle(canvas)) try: root.mainloop() finally: stop = 1 if __name__ == '__main__': main() ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] need info for externally maintained modules PEP
Phillip J. Eby wrote: > It would be good if we could have separate setup.py files for "external" > libraries, not only because of the ability to get PKG-INFO installed, but > also because then OS vendors that split those externals out into separate > system packages wouldn't need to jump through as many hoops to build just a > single external library like ElementTree or ctypes. -1. These aren't external libraries; they are part of Python. There are many build options (such as linking them statically through Modules/Setup, or building them as dynamic libraries using Modules/Setup, or building them through setup.py); adding more files would increase confusion. If you want additional files generated and installed, additional code can be put into setup.py to generate them. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Subversion repository back up
I just completed the repository import; write access to the projects repository is back. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] need info for externally maintained modules PEP
At 07:56 PM 4/9/2006 +0200, Martin v. Löwis wrote: >Phillip J. Eby wrote: > > It would be good if we could have separate setup.py files for "external" > > libraries, not only because of the ability to get PKG-INFO installed, but > > also because then OS vendors that split those externals out into separate > > system packages wouldn't need to jump through as many hoops to build > just a > > single external library like ElementTree or ctypes. > >-1. These aren't external libraries; they are part of Python. They *were* external libraries. Also, many OS vendors nonetheless split the standard library into different system packages, e.g. Debian's longstanding tradition of excising the distutils into a separate python-dev package. As much as we might wish that vendors not do these things, they often have practical matters of continuity and documentation to deal with; if they currently have a "python-ctypes" package, for example, they may wish to maintain that even when ctypes is bundled with 2.5. > There are >many build options (such as linking them statically through >Modules/Setup, or building them as dynamic libraries using >Modules/Setup, or building them through setup.py); adding more files >would increase confusion. I was hoping that we had reached a point where setup.py could simply become the One Obvious Way to do it, in which case the One Obvious Way to bundle formerly-external libraries would be to bundle their setup scripts. This is particularly helpful for external libraries that still maintain an external distribution for older Python versions. >If you want additional files generated and installed, additional code >can be put into setup.py to generate them. Of course it can, and that will certainly solve the issues for packages with dependencies on ctypes, ElementTree, and setuptools. But I was hoping for a more modular approach, one that would be more amenable to the needs of external library maintainers, and of system packagers who split even the core stdlib into multiple packages. Those packagers will have to figure out what parts of the new setup.py to extract and what to keep in order to do their splitting. And the people who develop the libraries will have to maintain two unrelated setup.py files. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] need info for externally maintained modules PEP
Phillip J. Eby wrote: > I was hoping that we had reached a point where setup.py could simply > become the One Obvious Way to do it, in which case the One Obvious Way > to bundle formerly-external libraries would be to bundle their setup > scripts. Well, setup.py cannot (yet?) replace the other mechanisms. It is certainly the default, yes. > Of course it can, and that will certainly solve the issues for packages > with dependencies on ctypes, ElementTree, and setuptools. But I was > hoping for a more modular approach, one that would be more amenable to > the needs of external library maintainers, and of system packagers who > split even the core stdlib into multiple packages. Those packagers will > have to figure out what parts of the new setup.py to extract and what to > keep in order to do their splitting. And the people who develop the > libraries will have to maintain two unrelated setup.py files. I can't envision how that would work, or what problems it would solve. Hence, I personally can't work on any of this. System packagers already have mechanisms for splitting the Python distribution into several packages. They typically run the entire build process, building all modules, and then put the various files into different packages. I don't think they have ever asked for better support of this process, and I doubt that additional setup.py will help them - they just apply the process that the where using all the time to the new release, and it will continue to work fine. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] int()'s ValueError behaviour
Guido van Rossum wrote:
> Go ahead and fix it. This was probably never changed since 1990 or
> so... Do expect some code brakage where people rely on the old
> behavior. :-(
>
> --Guido
>
> On 4/9/06, Thomas Wouters <[EMAIL PROTECTED]> wrote:
>> Someone on IRC (who refuses to report bugs on sourceforge, so I guess he
>> wants to remain anonymous) came with this very amusing bug: int(), when
>> raising ValueError, doesn't quote (or repr(), rather) its arguments:
>>
>> >>> int("")
>> Traceback (most recent call last):
>> File "", line 1, in ?
>> ValueError: invalid literal for int():
> int("34\n\n\n5")
>> Traceback (most recent call last):
>> File "", line 1, in ?
>> ValueError: invalid literal for int(): 34
>>
>>
>> 5
>> Unicode behaviour also isn't always consistent:
> int(u'\u0100')
>> Traceback (most recent call last):
>> File "", line 1, in ?
>> UnicodeEncodeError: 'decimal' codec can't encode character u'\u0100' in
>> position 0: invalid decimal Unicode string
> int(u'\u09ec', 6)
>> Traceback (most recent call last):
>> File "", line 1, in ?
>> ValueError: invalid literal for int(): 6
>>
>> And trying to use the 'decimal' codec directly:
> u'6'.encode('decimal')
>> Traceback (most recent call last):
>> File "", line 1, in ?
>> LookupError: unknown encoding: decimal
This part I can explain: the internal decimal codec isn't
made public through the codec registry since it only
supports encoding.
The encoder converts a Unicode decimal strings to plain
ASCII decimals.
The error message looks like a standard codec error message
because the raise_encode_exception() API is used.
>> I'm not sure if the latter problems are fixable, but the former should be
>> fixed by passing the argument to ValueError through repr(), I think. It's
>> also been suggested (by the reporter, and I agree) that the actual base
>> should be in the errormessage too. Is there some reason not to do this that
>> I've overlooked?
>>
>> --
>> Thomas Wouters <[EMAIL PROTECTED]>
>>
>> Hi! I'm a .signature virus! copy me into your .signature file to help me
>> spread!
>> ___
>> Python-Dev mailing list
>> [email protected]
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>>
>>
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/mal%40egenix.com
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Apr 09 2006)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free !
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] need info for externally maintained modules PEP
On Sun, Apr 09, 2006 at 02:48:47PM -0400, Phillip J. Eby wrote: > At 07:56 PM 4/9/2006 +0200, Martin v. Löwis wrote: [...] > >-1. These aren't external libraries; they are part of Python. > > They *were* external libraries. Also, many OS vendors nonetheless split > the standard library into different system packages, e.g. Debian's > longstanding tradition of excising the distutils into a separate python-dev > package. Debian has fixed this bug. The entire standard library, including distutils, is now in the python2.4 package. The python2.4-dev package mainly contains header files now. This has been true for some time; it's certainly the case in the current stable release. > As much as we might wish that vendors not do these things, they often have > practical matters of continuity and documentation to deal with; if they > currently have a "python-ctypes" package, for example, they may wish to > maintain that even when ctypes is bundled with 2.5. They can do that just by shipping an empty "python-ctypes" package that depends on the full python package. -Andrew. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] need info for externally maintained modules PEP
On Monday 10 April 2006 04:48, Phillip J. Eby wrote: > They *were* external libraries. Also, many OS vendors nonetheless > split the standard library into different system packages, e.g. > Debian's longstanding tradition of excising the distutils into a > separate python-dev package. Ubuntu (a debian derivative) and I _think_ Debian proper has fixed this now. Well, I'd be suprised if Debian proper hasn't fixed it as well, as the same person packages both. Anthony -- Anthony Baxter <[EMAIL PROTECTED]> It's never too late to have a happy childhood. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] need info for externally maintained modules PEP
At 09:35 AM 4/10/2006 +1000, Andrew Bennetts wrote: >On Sun, Apr 09, 2006 at 02:48:47PM -0400, Phillip J. Eby wrote: > > At 07:56 PM 4/9/2006 +0200, Martin v. Löwis wrote: >[...] > > >-1. These aren't external libraries; they are part of Python. > > > > They *were* external libraries. Also, many OS vendors nonetheless split > > the standard library into different system packages, e.g. Debian's > > longstanding tradition of excising the distutils into a separate > python-dev > > package. > >Debian has fixed this bug. And there was much rejoicing. :) > > As much as we might wish that vendors not do these things, they often have > > practical matters of continuity and documentation to deal with; if they > > currently have a "python-ctypes" package, for example, they may wish to > > maintain that even when ctypes is bundled with 2.5. > >They can do that just by shipping an empty "python-ctypes" package that >depends >on the full python package. Yeah, but why do something that logical and simple when you can create elaborate patches to remove functionality from setup.py? ;) But you've convinced me. I'd still prefer we generate these packages' PKG-INFO from the "formerly external" packages' setup.py files in order to ensure the metadata is correct. But if we have to do it manually, we have to do it manually. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] segfault (double free?) when '''-string crosses line
On Linux, In HEAD 2.5, but only with the non-debug version, I get a segfault when I do this: >>> ''' ... ''' It seems to occur for any triple-quoted string crossing a line boundary. A bit of the stack trace: #0 0x40030087 in pthread_mutex_lock () from /lib/i686/libpthread.so.0 #1 0x4207ad18 in free () from /lib/i686/libc.so.6 #2 0x08057990 in tok_nextc (tok=0x81c71d8) at ../Parser/tokenizer.c:809 #3 0x0805872d in tok_get (tok=0x81c71d8, p_start=0xb338, p_end=0xb33c) at ../Parser/tokenizer.c:1411 #4 0x08059042 in PyTokenizer_Get (tok=0x81c71d8, p_start=0xb338, p_end=0xb33c) at ../Parser/tokenizer.c:1514 #5 0x080568a7 in parsetok (tok=0x81c71d8, g=0x814a000, start=256, err_ret=0xb3a0, flags=0) at ../Parser/parsetok.c:135 Does this ring a bell? Is there already an SF bug open perhaps? On OSX, I get an interesting error: python2.5(12998) malloc: *** Deallocation of a pointer not malloced: 0x36b460; This could be a double free(), or free() called with the middle of an allocated block; Try setting environment variable MallocHelp to see tools to help debug -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] segfault (double free?) when '''-string crosses line
[Guido] > On Linux, In HEAD 2.5, but only with the non-debug version, I get a > segfault when I do this: > > >>> ''' > ... ''' > > It seems to occur for any triple-quoted string crossing a line > boundary. A bit of the stack trace: > > #0 0x40030087 in pthread_mutex_lock () from /lib/i686/libpthread.so.0 > #1 0x4207ad18 in free () from /lib/i686/libc.so.6 > #2 0x08057990 in tok_nextc (tok=0x81c71d8) at ../Parser/tokenizer.c:809 > #3 0x0805872d in tok_get (tok=0x81c71d8, p_start=0xb338, > p_end=0xb33c) > at ../Parser/tokenizer.c:1411 > #4 0x08059042 in PyTokenizer_Get (tok=0x81c71d8, p_start=0xb338, > p_end=0xb33c) at ../Parser/tokenizer.c:1514 > #5 0x080568a7 in parsetok (tok=0x81c71d8, g=0x814a000, start=256, > err_ret=0xb3a0, flags=0) at ../Parser/parsetok.c:135 > > > Does this ring a bell? Is there already an SF bug open perhaps? > > On OSX, I get an interesting error: > > python2.5(12998) malloc: *** Deallocation of a pointer not malloced: > 0x36b460; This could be a double free(), or free() called with the > middle of an allocated block; Try setting environment variable > MallocHelp to see tools to help debug It rings a bell here only in that the front end had lots of allocate-versus-free mismatches between the PyObject_ and PyMem_ raw-memory APIs, and this kind of failure smells a lot like that. For example, the ../Parser/tokenizer.c:809 in the traceback is PyMem_FREE(new); and _one_ way to set `new` is from the earlier well-hidden else if (tok_stdin_decode(tok, &new) != 0) where tok_stdin_decode() can do PyMem_FREE(*inp); *inp = converted; where `inp` is its local name for `new`, and `converted` comes from converted = new_string(PyString_AS_STRING(utf8), PyString_GET_SIZE(utf8)); and new_string() starts with char* result = (char *)PyObject_MALLOC(len + 1); So that's a mismatch, although I don't know whether it's the one that's triggering. When I repaired all the mismatches that caused tests to crash on my box, I changed affected front-end string mucking to use PyObject_ uniformly (strings are usual small, the small-object allocator is usually faster than the platform malloc, and half (exactly half :-) of the crash-causing mismatched pairs were using PyObject_ anyway). Someone who understands their way through the sub-maze above is encouraged to do the same for it. BTW, your "but only with the non-debug version" is more evidence: in a debug build, PyMem_ and PyObject_ calls are all redirected to Python's obmalloc, to take advantage of its debug-build padding gimmicks. It's only in a release build that PyMem_ resolves directly to the platform malloc/realloc/free. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] segfault (double free?) when '''-string crosses line
On 4/9/06, Tim Peters <[EMAIL PROTECTED]> wrote: > [Guido] > > On Linux, In HEAD 2.5, but only with the non-debug version, I get a > > segfault when I do this: > > > > >>> ''' > > ... ''' > > It rings a bell here only in that the front end had lots of > allocate-versus-free mismatches between the PyObject_ and PyMem_ > raw-memory APIs, and this kind of failure smells a lot like that. http://python.org/sf/1467512 fixes the problem for me on linux. It converts all the PyMem_* APIs to PyObject_* APIs. Assigned to Guido until he changes that. :-) There are several more places in the core that should probably use PyObject_* memory APIs since the alloced memory is small. 412 uses of PyMem_* in */*.c. Most of those are in modules where it is probably appropriate. But PyFutureFeatures could really use PyObject_* given it's only 8 bytes. (Python/future.c and Python/compile.c). Modules/_bsddb.c has a scary line: #define PyObject_Del PyMem_DEL n ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] segfault (double free?) when '''-string crosses line
[Neal Norwitz] > http://python.org/sf/1467512 fixes the problem for me on linux. It > converts all the PyMem_* APIs to PyObject_* APIs. Assigned to Guido > until he changes that. :-) Thanks! I didn't take that route, instead not changing anything I didn't take the time to understand first. For example, it's a pure loss to change tok_new() to use PyObject_ (as the patch does), since a struct tok_state is far too big for obmalloc to handle (e.g., the `indstack` member alone is too big -- that's an array of 100 ints, so consumes at least 400 bytes by itself). Using PyObject_ here just wastes some time figuring out that tok_state is too big for it, and passes the calls on to the system malloc/free. I thought about switching the readline thingies at the time, but quickly hit a wall: I have no idea where the source for #ifdef __VMS extern char* vms__StdioReadline(FILE *sys_stdin, FILE *sys_stdout, char *prompt); #endif may be (it's apparently not checked in, and Google couldn't find it either). Does anyone still work on the VMS port? If not, screw it ;-) > There are several more places in the core that should probably use > PyObject_* memory APIs since the alloced memory is small. 412 uses of > PyMem_* in */*.c. Most of those are in modules where it is probably > appropriate. But PyFutureFeatures could really use PyObject_* given > it's only 8 bytes. (Python/future.c and Python/compile.c). If we allocate one of those per PyAST_Compile, I'm not sure the difference would be measurable ;-) > Modules/_bsddb.c has a scary line: #define PyObject_Del PyMem_DEL That one's fine -- it's protected by an appropriate #ifdef: #if PYTHON_API_VERSION <= 1007 /* 1.5 compatibility */ #define PyObject_New PyObject_NEW #define PyObject_Del PyMem_DEL #endif The name "PyObject_Del" didn't exist in Pythons that old, so "something like that" is necessary to write C that works with all Pythons ever released. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] segfault (double free?) when '''-string crosses line
Tim Peters wrote: > #ifdef __VMS > extern char* vms__StdioReadline(FILE *sys_stdin, FILE *sys_stdout, > char *prompt); > #endif > > may be (it's apparently not checked in, and Google couldn't find it > either). Does anyone still work on the VMS port? If not, screw it > ;-) I would have no concerns with breaking the VMS port. If it is broken, we will either receive patches, or it stays broken. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
