Re: Python's garbage collection was Re: Python reliability

2005-10-14 Thread Patrick Down
Paul Rubin wrote: > I haven't been keeping up with this stuff in recent years so I have a > worse concern. I don't know whether it's founded or not. Basically > in the past decade or so, memory has gotten 100x larger and cpu's have > gotten 100x faster, but memory is less than 10x faster once yo

Re: Python's garbage collection was Re: Python reliability

2005-10-14 Thread [EMAIL PROTECTED]
Paul Rubin wrote: > This correctly describes difficulties of using a copying GC in > CPython. Note that the Boehm GC is mark-and-sweep. As Alex mentions, > that usually means there's a pause every so often while the GC scans > the entire heap, touching all data both live and dead (maybe the Boehm

Re: Python's garbage collection was Re: Python reliability

2005-10-14 Thread Mike Meyer
"Diez B. Roggisch" <[EMAIL PROTECTED]> writes: > And regarding the "zero exceptions" - I know for sure that quite a few > programs were crashing when the transition in 68K from 24 bit > addresses to real 32 bit was done on popular systems like the ATARI ST > - as some smart-asses back then used the

Re: Python's garbage collection was Re: Python reliability

2005-10-14 Thread Diez B. Roggisch
> Yeah, I noticed that, I could have been pedantic about it but chose to > just describe how these language implementations work in the real > world with zero exceptions that I know of. I guess I should have > spelled it out. You talked about CPU architectures: """ >And this presumes an archite

Re: Python's garbage collection was Re: Python reliability

2005-10-13 Thread Paul Rubin
[EMAIL PROTECTED] writes: > Folks, most common GC schemes have been tried as experiments over > the years. None have succeeeded, for various reasons. I think one > of the main reasons is that Python has to "play nice" with external > libraries, many of which weren't written with GC beyond malloc

Re: Python's garbage collection was Re: Python reliability

2005-10-13 Thread Paul Rubin
Steve Holden <[EMAIL PROTECTED]> writes: > >>And this presumes an architecture which byte-addresses and only > >> uses "aligned" addresses. > > He was talking about the arachiteecture, for Pete's sake, not a compiler. Yeah, I noticed that, I could have been pedantic about it but chose to just d

Re: Python's garbage collection was Re: Python reliability

2005-10-13 Thread Steve Holden
Paul Rubin wrote: > "Diez B. Roggisch" <[EMAIL PROTECTED]> writes: > >>>Yes, that would describe just about every cpu for the past 30 years >>>that's a plausible Python target. >> >>No. The later 68K (>68020) could address on odd adresses. And AFAIK >>all x86 can because of their 8080 stemming. >

Re: Python's garbage collection was Re: Python reliability

2005-10-13 Thread Paul Rubin
"Diez B. Roggisch" <[EMAIL PROTECTED]> writes: > > Yes, that would describe just about every cpu for the past 30 years > > that's a plausible Python target. > > No. The later 68K (>68020) could address on odd adresses. And AFAIK > all x86 can because of their 8080 stemming. Yes, "could" but not "

Re: Python's garbage collection was Re: Python reliability

2005-10-13 Thread Diez B. Roggisch
> Yes, that would describe just about every cpu for the past 30 years > that's a plausible Python target. No. The later 68K (>68020) could address on odd adresses. And AFAIK all x86 can because of their 8080 stemming. Don't confuse this with 16Bit aligned addressing - _that_ is the minimum for

Re: Python's garbage collection was Re: Python reliability

2005-10-13 Thread Paul Rubin
Scott David Daniels <[EMAIL PROTECTED]> writes: > > I think most of the time, branch prediction will prevent the cache > > flush. > But, branch prediction is usually a compiler thing, based on code > that is, in this case, a spot in the interpreter that is actually > taking both sides of the branch

Re: Python's garbage collection was Re: Python reliability

2005-10-13 Thread Scott David Daniels
Paul Rubin wrote: > Scott David Daniels <[EMAIL PROTECTED]> writes: > >>Current speeds are due to deep pipelines, and a conditional in the >>INCREF code would blow a pipeline. > > > I think most of the time, branch prediction will prevent the cache > flush. But, branch prediction is usually a co

Re: Python's garbage collection was Re: Python reliability

2005-10-13 Thread skip
Diez> AFAIK some LISPs do a similar trick to carry int values on Diez> cons-cells. And by this tehy reduce integer precision to 28 bit Diez> or something. Surely _not_ going to pass a regression test suite Diez> :) I'm pretty sure this was tried a few years ago w/ Python. I don'

Re: Python's garbage collection was Re: Python reliability

2005-10-13 Thread Paul Rubin
Scott David Daniels <[EMAIL PROTECTED]> writes: > Current speeds are due to deep pipelines, and a conditional in the > INCREF code would blow a pipeline. I think most of the time, branch prediction will prevent the cache flush. Anyway, with consed integers, there's still going to be a conditional

Re: Python's garbage collection was Re: Python reliability

2005-10-13 Thread Scott David Daniels
Paul Rubin wrote: > "Diez B. Roggisch" <[EMAIL PROTECTED]> writes: (about tag bits) >>... Basically I think that trying to come up with all sorts of >> optimizations for rather marginal problems (number crunching >> should be - if a python domain at all - done using Numarray) > I don't think

Re: Python's garbage collection was Re: Python reliability

2005-10-13 Thread Alex Martelli
Tom Anderson <[EMAIL PROTECTED]> wrote: > On Tue, 11 Oct 2005, Alex Martelli wrote: > > > Tom Anderson <[EMAIL PROTECTED]> wrote: > > ... > >> Has anyone looked into using a real GC for python? I realise it would be a > > > > If you mean mark-and-sweep, with generational twists, > > Yes, more

Re: Python's garbage collection was Re: Python reliability

2005-10-13 Thread Paul Rubin
Steve Holden <[EMAIL PROTECTED]> writes: > Until someone does the experiment this stuff is bound to be > speculation (what's that saying about "premature optimization"?). 40 years of practical Lisp implementation efforts and around the globe and hundreds of published papers on the subject might n

Re: Python's garbage collection was Re: Python reliability

2005-10-13 Thread Steve Holden
Paul Rubin wrote: > "Diez B. Roggisch" <[EMAIL PROTECTED]> writes: > >>That particular implementation used 3 or 4 tag-bits. Of course you are >>right that nowadays python won't notice the difference, as larger nums >>get implicitely converted to a suitable representation. But then the >>efficiency

Re: Python's garbage collection was Re: Python reliability

2005-10-13 Thread Paul Rubin
"Fredrik Lundh" <[EMAIL PROTECTED]> writes: > (fwiw, switching to tagging in CPython would break most about > everything. might as well start over, and nobody's likely to do > that to speed up integer- dominated programs a little...) Yeah, a change of that magnitude in CPython would be madness, b

Re: Python's garbage collection was Re: Python reliability

2005-10-13 Thread Paul Rubin
"Diez B. Roggisch" <[EMAIL PROTECTED]> writes: > That particular implementation used 3 or 4 tag-bits. Of course you are > right that nowadays python won't notice the difference, as larger nums > get implicitely converted to a suitable representation. But then the > efficiency goes away... Basically

Re: Python's garbage collection was Re: Python reliability

2005-10-13 Thread Fredrik Lundh
Tom Anderson wrote: > In both smalltalk and python, every single variable contains a reference > to an object - there isn't the object/primitive distinction you find in > less advanced languages like java. > > Except that in smalltalk, this isn't true: in ST, every variable *appears* > to contain

Re: Python's garbage collection was Re: Python reliability

2005-10-13 Thread Diez B. Roggisch
Paul Rubin wrote: > "Diez B. Roggisch" <[EMAIL PROTECTED]> writes: > >>AFAIK some LISPs do a similar trick to carry int values on >>cons-cells. And by this tehy reduce integer precision to 28 bit or >>something. Surely _not_ going to pass a regression test suite :) > > > Lisps often use just one

Re: Python's garbage collection was Re: Python reliability

2005-10-13 Thread Paul Rubin
"Diez B. Roggisch" <[EMAIL PROTECTED]> writes: > AFAIK some LISPs do a similar trick to carry int values on > cons-cells. And by this tehy reduce integer precision to 28 bit or > something. Surely _not_ going to pass a regression test suite :) Lisps often use just one tag bit, to distinguish betwe

Re: Python's garbage collection was Re: Python reliability

2005-10-13 Thread Diez B. Roggisch
Delaney, Timothy (Tim) wrote: > Tom Anderson wrote: > > >>Except that in smalltalk, this isn't true: in ST, every variable >>*appears* to contain a reference to an object, but implementations >>may not actually work like that. In particular, SmallTalk 80 (and >>some earlier smalltalks, and all su

Re: Python's garbage collection was Re: Python reliability

2005-10-12 Thread Tom Anderson
On Tue, 11 Oct 2005, Alex Martelli wrote: > Tom Anderson <[EMAIL PROTECTED]> wrote: > ... >> Has anyone looked into using a real GC for python? I realise it would be a > > If you mean mark-and-sweep, with generational twists, Yes, more or less. > that's what gc uses for cyclic garbage. Do you

Re: Python's garbage collection was Re: Python reliability

2005-10-12 Thread jepler
On Mon, Oct 10, 2005 at 08:37:03PM +0100, Tom Anderson wrote: > So python doesn't use the old SmallTalk 80 SmallInteger hack, or similar? > Fair enough - the performance gain is nice, but the extra complexity would > be a huge pain, i imagine. I tried to implement this once. There was not a per

RE: Python's garbage collection was Re: Python reliability

2005-10-12 Thread Delaney, Timothy (Tim)
Tom Anderson wrote: > Except that in smalltalk, this isn't true: in ST, every variable > *appears* to contain a reference to an object, but implementations > may not actually work like that. In particular, SmallTalk 80 (and > some earlier smalltalks, and all subsequent smalltalks, i think) > handl

Re: Python's garbage collection was Re: Python reliability

2005-10-12 Thread Tom Anderson
On Mon, 10 Oct 2005, it was written: > Tom Anderson <[EMAIL PROTECTED]> writes: > >> Has anyone looked into using a real GC for python? I realise it would >> be a lot more complexity in the interpreter itself, but it would be >> faster, more reliable, and would reduce the complexity of extension

Re: Python's garbage collection was Re: Python reliability

2005-10-12 Thread Tom Anderson
On Wed, 12 Oct 2005, Jorgen Grahn wrote: > On Mon, 10 Oct 2005 20:37:03 +0100, Tom Anderson <[EMAIL PROTECTED]> wrote: >> On Mon, 10 Oct 2005, it was written: > ... >>> There is no way you can avoid making garbage. Python conses everything, >>> even integers (small positive ones are cached). >> >

Re: Python's garbage collection was Re: Python reliability

2005-10-12 Thread Jorgen Grahn
On Mon, 10 Oct 2005 20:37:03 +0100, Tom Anderson <[EMAIL PROTECTED]> wrote: > On Mon, 10 Oct 2005, it was written: ... >> There is no way you can avoid making garbage. Python conses everything, >> even integers (small positive ones are cached). > > So python doesn't use the old SmallTalk 80 Small

Re: Python's garbage collection was Re: Python reliability

2005-10-11 Thread Paul Rubin
[EMAIL PROTECTED] (Alex Martelli) writes: > > Has anyone looked into using a real GC for python? ... > > lot more complexity in the interpreter itself, but it would be faster, > > more reliable, and would reduce the complexity of extensions. > > ??? It adds no complexity (it's already there), it'

Re: Python's garbage collection was Re: Python reliability

2005-10-11 Thread Alex Martelli
Tom Anderson <[EMAIL PROTECTED]> wrote: ... > Has anyone looked into using a real GC for python? I realise it would be a If you mean mark-and-sweep, with generational twists, that's what gc uses for cyclic garbage. > lot more complexity in the interpreter itself, but it would be faster, > more

Re: Python's garbage collection was Re: Python reliability

2005-10-10 Thread Paul Rubin
Tom Anderson <[EMAIL PROTECTED]> writes: > Has anyone looked into using a real GC for python? I realise it would > be a lot more complexity in the interpreter itself, but it would be > faster, more reliable, and would reduce the complexity of extensions. The next PyPy sprint (this week I think) is

Re: Python's garbage collection was Re: Python reliability

2005-10-10 Thread Mike Meyer
Tom Anderson <[EMAIL PROTECTED]> writes: > Has anyone looked into using a real GC for python? I realise it would > be a lot more complexity in the interpreter itself, but it would be > faster, more reliable, and would reduce the complexity of extensions. > > Hmm. Maybe it wouldn't make extensions e

Re: Python's garbage collection was Re: Python reliability

2005-10-10 Thread Aahz
In article <[EMAIL PROTECTED]>, Tom Anderson <[EMAIL PROTECTED]> wrote: > >Has anyone looked into using a real GC for python? I realise it would be a >lot more complexity in the interpreter itself, but it would be faster, >more reliable, and would reduce the complexity of extensions. > >Hmm. May