Re: Copy-on-write friendly Python garbage collection (Posting On Python-List Prohibited)

2018-01-01 Thread Niles Rogoff
> Niles, if you want to claim wxjmfauth is right, you'll have to present > some actual evidence.  He's claimed for years that Python's Unicode > support is buggy (as he does here), without ever demonstrating a bug. > We've long ago tired of trying to reason with him. > > The tradeoffs of memory u

Re: Copy-on-write friendly Python garbage collection (Posting On Python-List Prohibited)

2018-01-01 Thread Ned Batchelder
On 1/1/18 1:49 PM, Niles Rogoff wrote: On Mon, 01 Jan 2018 10:42:58 -0800, breamoreboy wrote: On Monday, January 1, 2018 at 10:14:59 AM UTC, wxjm...@gmail.com wrote: Le lundi 1 janvier 2018 08:35:53 UTC+1, Lawrence D’Oliveiro a écrit : On Monday, January 1, 2018 at 7:52:48 AM UTC+13, Paul Rub

Re: Copy-on-write friendly Python garbage collection

2018-01-01 Thread breamoreboy
On Monday, January 1, 2018 at 12:53:03 PM UTC, Wu Xi wrote: > breamoreboy: > > On Sunday, December 31, 2017 at 6:19:13 PM UTC, Wu Xi wrote: > >> breamoreboy: > >>> An interesting write up on something that is incorporated into Python 3.7 > >>> https://engineering.instagram.com/copy-on-write-friend

Re: Copy-on-write friendly Python garbage collection (Posting On Python-List Prohibited)

2018-01-01 Thread Niles Rogoff
On Mon, 01 Jan 2018 10:42:58 -0800, breamoreboy wrote: > On Monday, January 1, 2018 at 10:14:59 AM UTC, wxjm...@gmail.com wrote: >> Le lundi 1 janvier 2018 08:35:53 UTC+1, Lawrence D’Oliveiro a écrit : >> > On Monday, January 1, 2018 at 7:52:48 AM UTC+13, Paul Rubin wrote: >> > > I wonder if thing

Re: Copy-on-write friendly Python garbage collection

2018-01-01 Thread Wu Xi
breamore...@gmail.com: > On Sunday, December 31, 2017 at 6:19:13 PM UTC, Wu Xi wrote: >> breamoreboy: >>> An interesting write up on something that is incorporated into Python 3.7 >>> https://engineering.instagram.com/copy-on-write-friendly-python-garbage-collection-ad6ed5233ddf >> >> Appearantly,

Re: Copy-on-write friendly Python garbage collection

2018-01-01 Thread breamoreboy
On Sunday, December 31, 2017 at 6:19:13 PM UTC, Wu Xi wrote: > breamoreboy: > > An interesting write up on something that is incorporated into Python 3.7 > > https://engineering.instagram.com/copy-on-write-friendly-python-garbage-collection-ad6ed5233ddf > > Appearantly, Erlang is the way to go, w

Re: Copy-on-write friendly Python garbage collection

2018-01-01 Thread INADA Naoki
FYI: https://bugs.python.org/issue31558 INADA Naoki On Mon, Jan 1, 2018 at 12:39 AM, wrote: > An interesting write up on something that is incorporated into Python 3.7 > https://engineering.instagram.com/copy-on-write-friendly-python-garbage-collection-ad6ed5233ddf > > -- > Kindest regards.

Re: Copy-on-write friendly Python garbage collection

2017-12-31 Thread Wu Xi
breamore...@gmail.com: > An interesting write up on something that is incorporated into Python 3.7 > https://engineering.instagram.com/copy-on-write-friendly-python-garbage-collection-ad6ed5233ddf Appearantly, Erlang is the way to go, when it comes to web frameworks. -- https://mail.python.org/m

Re: Copy-on-write friendly Python garbage collection

2017-12-31 Thread Wu Xi
they said they run the largest deployment of Django world-wide. be it as it may... many still consider the web guys to be the "funny people". Why did they not switch over to Erlang ? -- https://mail.python.org/mailman/listinfo/python-list

Re: Copy-on-write friendly Python garbage collection

2017-12-31 Thread Wu Xi
breamore...@gmail.com: > An interesting write up on something that is incorporated into Python 3.7 > https://engineering.instagram.com/copy-on-write-friendly-python-garbage-collection-ad6ed5233ddf kewl py 3.7 does not fully make install here, but it built and works, as far as I can tell -- htt

Re: copy on write

2012-02-06 Thread Wolfram Hinderer
On 3 Feb., 11:47, John O'Hagan wrote: > But isn't it equally true if we say that z = t[1], then t[1] += x is > syntactic sugar for z = z.__iadd__(x)? Why should that fail, if z can handle > it? It's more like syntactic sugar for y = t; z = y.__getitem__(1); z.__iadd__(x); y.__setitem__(1, z)

Re: copy on write

2012-02-06 Thread Rick Johnson
On Feb 2, 11:28 pm, Chris Angelico wrote: > On Fri, Feb 3, 2012 at 4:04 PM, Steven D'Aprano > > wrote: > > No matter what Python did, somebody would complain. > > +1 > > This is, I think, the ultimate truth of the matter. People would not complain if they did not care. The only useless complaint

Re: copy on write

2012-02-03 Thread 88888 Dihedral
在 2012年1月14日星期六UTC+8上午6时48分29秒,Evan Driscoll写道: > On 01/13/2012 03:20 PM, Neil Cerutti wrote: > > They perform the same action, but their semantics are different. > > operator+ will always return a new object, thanks to its > > signature, and operator+= shall never do so. That's the main > > differ

Re: copy on write

2012-02-03 Thread OKB (not okblacke)
Steven D'Aprano wrote: > Perhaps lists shouldn't define += at all, but then people will > complain that mylist += another_list is slow. Telling them to use > mylist.extend instead just makes them cranky. After all, mylist + > another_list works, so why shouldn't += work? It would work, it

Re: copy on write

2012-02-03 Thread John O'Hagan
On 03 Feb 2012 05:04:39 GMT Steven D'Aprano wrote: > On Fri, 03 Feb 2012 14:08:06 +1100, John O'Hagan wrote: > > > I think we're 12 years late on this one. It's PEP 203 from 2000 and > > the key phrase was: > > > > "The in-place function should always return a new reference, either > > to the o

Re: copy on write

2012-02-03 Thread Antoon Pardon
On 02/03/2012 06:04 AM, Steven D'Aprano wrote: Ultimately, there is no right answer, because the multitude of requirements are contradictory. No matter what Python did, somebody would complain. Which makes me wonder why it was introduced at all, or at least so fast If you see the difference in s

Re: copy on write

2012-02-02 Thread Chris Angelico
On Fri, Feb 3, 2012 at 4:04 PM, Steven D'Aprano wrote: > No matter what Python did, somebody would complain. +1 This is, I think, the ultimate truth of the matter. ChrisA -- http://mail.python.org/mailman/listinfo/python-list

Re: copy on write

2012-02-02 Thread Steven D'Aprano
On Fri, 03 Feb 2012 14:08:06 +1100, John O'Hagan wrote: > I think we're 12 years late on this one. It's PEP 203 from 2000 and the > key phrase was: > > "The in-place function should always return a new reference, either to > the old `x' object if the operation was indeed performed in-place, or to

Re: copy on write

2012-02-02 Thread John O'Hagan
On Thu, 02 Feb 2012 12:25:00 -0500 Terry Reedy wrote: > On 2/2/2012 9:17 AM, John O'Hagan wrote: > > > It's not so much about the type of x but that of x[1]. Wouldn't it > > be possible to omit the assignment simply if the object referred to > > by x[1] uses "+=" without creating a new object? T

Re: copy on write

2012-02-02 Thread Evan Driscoll
On 01/-10/-28163 01:59 PM, 8 Dihedral wrote: If you're working in C++ and overload your operators so that 'a +=' and 'a = + b' have different observable behaviors (besides perhaps time), then either your implementation is buggy or your design is very bad-mannered. Evan Do you mean the resu

Re: copy on write

2012-02-02 Thread Terry Reedy
On 2/2/2012 9:17 AM, John O'Hagan wrote: It's not so much about the type of x but that of x[1]. Wouldn't it be possible to omit the assignment simply if the object referred to by x[1] uses "+=" without creating a new object? That way, some_tuple[i] += y will succeed if some_tuple[i] is a list bu

Re: copy on write

2012-02-02 Thread Devin Jeanpierre
On Thu, Feb 2, 2012 at 11:28 AM, MRAB wrote: > Should failed assignment be raising TypeError? Is it really a type > error? A failed setitem should be a TypeError as much as a failed getitem should. Should 1[0] be a TypeError? -- Devin -- http://mail.python.org/mailman/listinfo/python-list

Re: copy on write

2012-02-02 Thread MRAB
On 02/02/2012 10:53, Hrvoje Niksic wrote: Steven D'Aprano writes: Perhaps you are thinking that Python could determine ahead of time whether x[1] += y involved a list or a tuple, and not perform the finally assignment if x was a tuple. Well, maybe, but such an approach (if possible!) is fr

Re: copy on write

2012-02-02 Thread John O'Hagan
On 02 Feb 2012 09:16:40 GMT Steven D'Aprano wrote: > On Thu, 02 Feb 2012 19:11:53 +1100, John O'Hagan wrote: > > > You're right, in fact, for me the surprise is that "t[1] +=" is > > interpreted as an assignment at all, given that for lists (and other > > mutable objects which use "+=") it is a

Re: copy on write

2012-02-02 Thread 88888 Dihedral
在 2012年1月14日星期六UTC+8上午6时48分29秒,Evan Driscoll写道: > On 01/13/2012 03:20 PM, Neil Cerutti wrote: > > They perform the same action, but their semantics are different. > > operator+ will always return a new object, thanks to its > > signature, and operator+= shall never do so. That's the main > > differ

Re: copy on write

2012-02-02 Thread Hrvoje Niksic
Steven D'Aprano writes: > Perhaps you are thinking that Python could determine ahead of time > whether x[1] += y involved a list or a tuple, and not perform the > finally assignment if x was a tuple. Well, maybe, but such an approach > (if possible!) is fraught with danger and mysterious errors e

Re: copy on write

2012-02-02 Thread Thomas Rachel
Am 13.01.2012 13:30 schrieb Chris Angelico: It seems there's a distinct difference between a+=b (in-place addition/concatenation) and a=a+b (always rebinding), There is indeed. a = a + b is a = a.__add__(b), while a += b is a = a.__iadd__(b). __add__() is supposed to leave the original obje

Re: copy on write

2012-02-02 Thread Steven D'Aprano
On Thu, 02 Feb 2012 19:11:53 +1100, John O'Hagan wrote: > You're right, in fact, for me the surprise is that "t[1] +=" is > interpreted as an assignment at all, given that for lists (and other > mutable objects which use "+=") it is a mutation. Although as Steven > says elsewhere, it actually is a

Re: copy on write

2012-02-02 Thread John O'Hagan
On Thu, 2 Feb 2012 01:34:48 -0500 Devin Jeanpierre wrote: > On Wed, Feb 1, 2012 at 10:18 PM, John O'Hagan > wrote: > > On Fri, 13 Jan 2012 10:40:47 -0800 > > Ethan Furman wrote: > > > >> Steven D'Aprano wrote: > >> > Normally this is harmless, but there is one interesting little > >> > glitch y

Re: copy on write

2012-02-01 Thread Devin Jeanpierre
On Wed, Feb 1, 2012 at 10:18 PM, John O'Hagan wrote: > On Fri, 13 Jan 2012 10:40:47 -0800 > Ethan Furman wrote: > >> Steven D'Aprano wrote: >> > Normally this is harmless, but there is one interesting little >> > glitch you can get: >> > >> t = ('a', [23]) >> t[1] += [42] >> > Traceback

Re: copy on write

2012-02-01 Thread Steven D'Aprano
On Wed, 01 Feb 2012 19:51:13 -0800, Rick Johnson wrote: > Yeah there's a word for that; INTUITIVE, And I've been preaching its > virtues (sadly in vain it seems!) to these folks for some time now. Intuitive to whom? Expert Python programmers? VB coders? Perl hackers? School children who have

Re: copy on write

2012-02-01 Thread Rick Johnson
On Jan 13, 10:48 am, Devin Jeanpierre wrote: > On Fri, Jan 13, 2012 at 10:13 AM, Grant Edwards > wrote: > > On 2012-01-13, Devin Jeanpierre wrote: > >> On Fri, Jan 13, 2012 at 7:30 AM, Chris Angelico wrote: > There's a bit of a feeling > that code should "do what it looks like" and be sort of

Re: copy on write

2012-02-01 Thread John O'Hagan
On Fri, 13 Jan 2012 10:40:47 -0800 Ethan Furman wrote: > Steven D'Aprano wrote: > > Normally this is harmless, but there is one interesting little > > glitch you can get: > > > t = ('a', [23]) > t[1] += [42] > > Traceback (most recent call last): > > File "", line 1, in > > TypeErro

Re: copy on write

2012-01-13 Thread Evan Driscoll
On 01/13/2012 03:20 PM, Neil Cerutti wrote: They perform the same action, but their semantics are different. operator+ will always return a new object, thanks to its signature, and operator+= shall never do so. That's the main difference I was getting at. I was talking about the combination of

Re: copy on write

2012-01-13 Thread 88888 Dihedral
Ethan Furman於 2012年1月14日星期六UTC+8上午2時40分47秒寫道: > Steven D'Aprano wrote: > > Normally this is harmless, but there is one interesting little glitch you > > can get: > > > t = ('a', [23]) > t[1] += [42] > > Traceback (most recent call last): > > File "", line 1, in > > TypeError: 'tuple'

Re: copy on write

2012-01-13 Thread Neil Cerutti
On 2012-01-13, Evan Driscoll wrote: > On 01/13/2012 10:54 AM, Neil Cerutti wrote: >> If you've ever implemented operator=, operator+, and operator+= >> in C++ you'll know how and why they are different. > > At the same time, you'd also know that that implementing them > in such a way that 'a += b'

Re: copy on write

2012-01-13 Thread Evan Driscoll
On 01/13/2012 10:54 AM, Neil Cerutti wrote: If you've ever implemented operator=, operator+, and operator+= in C++ you'll know how and why they are different. At the same time, you'd also know that that implementing them in such a way that 'a += b' does *not* perform the same action as 'a = a

Re: copy on write

2012-01-13 Thread Neil Cerutti
On 2012-01-13, Grant Edwards wrote: > On 2012-01-13, Chris Angelico wrote: >> On Sat, Jan 14, 2012 at 5:15 AM, Grant Edwards >> wrote: >>> That assumes that C++ programmers understand C++. >> >> I understand C++ very well. That's why I use Python or Pike. >> >> (With apologies to Larry Wall) > >

Re: copy on write

2012-01-13 Thread Ethan Furman
Steven D'Aprano wrote: Normally this is harmless, but there is one interesting little glitch you can get: t = ('a', [23]) t[1] += [42] Traceback (most recent call last): File "", line 1, in TypeError: 'tuple' object does not support item assignment t ('a', [23, 42]) There is one other

Re: copy on write

2012-01-13 Thread Grant Edwards
On 2012-01-13, Chris Angelico wrote: > On Sat, Jan 14, 2012 at 5:15 AM, Grant Edwards > wrote: >> That assumes that C++ programmers understand C++. > > I understand C++ very well. That's why I use Python or Pike. > > (With apologies to Larry Wall) Were one inclined to troll a bit, one might be

Re: copy on write

2012-01-13 Thread Chris Angelico
On Sat, Jan 14, 2012 at 5:15 AM, Grant Edwards wrote: > That assumes that C++ programmers understand C++. I understand C++ very well. That's why I use Python or Pike. (With apologies to Larry Wall) ChrisA -- http://mail.python.org/mailman/listinfo/python-list

Re: copy on write

2012-01-13 Thread Grant Edwards
On 2012-01-13, Neil Cerutti wrote: > If you've ever implemented operator=, operator+, and operator+= > in C++ you'll know how and why they are different. That assumes that C++ programmers understand C++. ;) > A C++ programmer would be wondering how either can work on immutable > objects, and t

Re: copy on write

2012-01-13 Thread Neil Cerutti
On 2012-01-13, Devin Jeanpierre wrote: > On Fri, Jan 13, 2012 at 10:13 AM, Grant Edwards > wrote: >> On 2012-01-13, Devin Jeanpierre wrote: >>> On Fri, Jan 13, 2012 at 7:30 AM, Chris Angelico wrote: It seems there's a distinct difference between a+=b (in-place addition/concatenation)

Re: copy on write

2012-01-13 Thread Devin Jeanpierre
On Fri, Jan 13, 2012 at 10:13 AM, Grant Edwards wrote: > On 2012-01-13, Devin Jeanpierre wrote: >> On Fri, Jan 13, 2012 at 7:30 AM, Chris Angelico wrote: >>> It seems there's a distinct difference between a+=b (in-place >>> addition/concatenation) and a=a+b (always rebinding), which is sorely >>

Re: copy on write

2012-01-13 Thread Grant Edwards
On 2012-01-13, Devin Jeanpierre wrote: > On Fri, Jan 13, 2012 at 7:30 AM, Chris Angelico wrote: >> It seems there's a distinct difference between a+=b (in-place >> addition/concatenation) and a=a+b (always rebinding), which is sorely >> confusing to C programmers. But then, there's a lot about Py

Re: copy on write

2012-01-13 Thread Devin Jeanpierre
On Fri, Jan 13, 2012 at 7:30 AM, Chris Angelico wrote: > It seems there's a distinct difference between a+=b (in-place > addition/concatenation) and a=a+b (always rebinding), which is sorely > confusing to C programmers. But then, there's a lot about Python > that's sorely confusing to C programme

Re: copy on write

2012-01-13 Thread Steven D'Aprano
On Fri, 13 Jan 2012 23:30:56 +1100, Chris Angelico wrote: > It seems there's a distinct difference between a+=b (in-place > addition/concatenation) and a=a+b (always rebinding), Actually, both are always rebinding. It just happens that sometimes a+=b rebinds to the same object that it was origi

Re: copy on write

2012-01-13 Thread Chris Angelico
On Fri, Jan 13, 2012 at 11:10 PM, Steven D'Aprano wrote: z = [x, y]  # z is a list containing the same sublist twice z[0].append(23) print z > [[42, 23], [42, 23]] > > When you work with floats, ints or strings, you don't notice this because > those types are immutable: you can't mo

Re: copy on write

2012-01-13 Thread Steven D'Aprano
On Fri, 13 Jan 2012 11:33:24 +, Eduardo Suarez-Santana wrote: > I wonder whether this is normal behaviour. > > I would expect equal sign to copy values from right to left. Assignment in Python never copies values. > However, it > seems there is a copy-on-write mechanism that is not working

Re: copy on write

2012-01-13 Thread Jean-Michel Pichavant
Eduardo Suarez-Santana wrote: El 13/01/12 11:33, Eduardo Suarez-Santana escribió: I wonder whether this is normal behaviour. Even simpler: $ python Python 2.7.2 (default, Oct 31 2011, 11:54:55) [GCC 4.5.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> r=

Re: copy on write

2012-01-13 Thread Eduardo Suarez-Santana
El 13/01/12 11:33, Eduardo Suarez-Santana escribió: I wonder whether this is normal behaviour. Even simpler: $ python Python 2.7.2 (default, Oct 31 2011, 11:54:55) [GCC 4.5.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> r={'a':1}; >>> d={}; >>> d['x']=r

Re: Copy-on-write when forking a python process

2011-04-12 Thread jac
Heiko, Thank you for pointing out POSH. I have used some of python's other shared memory facilities, but was completely unaware of POSH, it seems nice. Also, I agree that shared memory would solve the use-case I outlined above, but it is not hard to imagine a slightly different case where the chil

Re: Copy-on-write when forking a python process

2011-04-08 Thread Heiko Wundram
Am 08.04.2011 20:34, schrieb jac: > I disagree with your statement that COW is an optimization for a > complete clone, it is an optimization that works at the memory page > level, not at the memory image level. In other words, if I write to a > copy-on-write page, only that page is copied into my

Re: Copy-on-write when forking a python process

2011-04-08 Thread jac
Hi Heiko, I just realized I should probably have put a clearer use-case in my previous message. A example use-case would be if you have a parent process which creates a large dictionary (say several gigabytes). The process then forks several worker processes which access this dictionary. The wor

Re: Copy-on-write when forking a python process

2011-04-08 Thread Heiko Wundram
Am 08.04.2011 18:14, schrieb John Connor: > Has anyone else looked into the COW problem? Are there workarounds > and/or other plans to fix it? Does the solution I am proposing sound > reasonable, or does it seem like overkill? Does anyone foresee any > problems with it? Why'd you need a "fix" l