[issue9942] Allow memory sections to be OS MERGEABLE

2010-09-25 Thread Kevin Hunter
Kevin Hunter added the comment: > Why do you read it into a cStringIO? A cStringIO has the same interface > as a file, so you could simply operate on the file directly. In that particular case, because it isn't actually a file. That workflow was my attempt at simplification to i

[issue9942] Allow memory sections to be OS MERGEABLE

2010-09-25 Thread Kevin Hunter
Kevin Hunter added the comment: > Well, first, this would only work for large objects. [...] > Why do you think you might have such duplication in your workload? Some of the projects with which I work involve multiple manipulations of large datasets. Often, we use Python scripts as

[issue9942] Allow memory sections to be OS MERGEABLE

2010-09-24 Thread Kevin Hunter
Kevin Hunter added the comment: My first thought is "Why is the reference counter stored with the object itself?" I imagine there are very good reasons, however, and this is not an area in which I have much mastery. Answering the question as best I can: I don't know ho

[issue9942] Allow memory sections to be OS MERGEABLE

2010-09-24 Thread Kevin Hunter
New submission from Kevin Hunter : Should Python enable a way for folks to inform the OS of MADV_MERGEABLE memory? I can't speak for other OSs, but Linux added the ability for processes to inform the kernel that they have memory that will likely not change for a while in 2.6.32. This is