Re: array.array()'s memory shared with multiprocessing.Process()

2017-09-12 Thread Stephan Houben
Op 2017-09-12, Thomas Jollans schreef : > I'm not sure actually. Maybe someone else here can help. I have a hunch > that on Windows the memory might not be shared in the same way that it > would on Linux/Unix, since Windows simply doesn't have the same process > forking capabilities as Unix. `mul

Re: array.array()'s memory shared with multiprocessing.Process()

2017-09-12 Thread Thomas Jollans
On 12/09/17 10:21, gerlando.fala...@gmail.com wrote: > Il giorno lunedì 11 settembre 2017 12:19:27 UTC+2, Thomas Jollans ha scritto: >> On 2017-09-10 23:05, iurly wrote: >>> As far as I'm concerned, I'm probably better off using double buffers to >>> avoid this kind of issues. >>> Thanks a lot for

Re: array.array()'s memory shared with multiprocessing.Process()

2017-09-12 Thread Stephan Houben
Op 2017-09-12, gerlando.fala...@gmail.com schreef : > Notice however how I'd have to create those Arrays dynamically in the > producer thread. Would I then be able to pass them to the consumer by > putting a reference in a queue? Yes. > I wouldn't want them to be pickled at all in that case, of

Re: array.array()'s memory shared with multiprocessing.Process()

2017-09-12 Thread gerlando . falauto
Il giorno lunedì 11 settembre 2017 12:19:27 UTC+2, Thomas Jollans ha scritto: > On 2017-09-10 23:05, iurly wrote: > > As far as I'm concerned, I'm probably better off using double buffers to > > avoid this kind of issues. > > Thanks a lot for your help! > > > > > That should work. Some other th

Re: array.array()'s memory shared with multiprocessing.Process()

2017-09-11 Thread Thomas Jollans
On 2017-09-10 23:05, iurly wrote: > As far as I'm concerned, I'm probably better off using double buffers to > avoid this kind of issues. > Thanks a lot for your help! > That should work. Some other things to consider, if both processes are on the same machine, are a series of memory-mapped fil

Re: array.array()'s memory shared with multiprocessing.Process()

2017-09-10 Thread Terry Reedy
On 9/10/2017 5:05 PM, iurly wrote: Il giorno domenica 10 settembre 2017 18:53:33 UTC+2, MRAB ha scritto: I've had a quick look at the source code. When an object is put into the queue, it's actually put into an internal buffer (a deque), and then the method returns. An internal thread works

Re: array.array()'s memory shared with multiprocessing.Process()

2017-09-10 Thread iurly
Il giorno domenica 10 settembre 2017 18:53:33 UTC+2, MRAB ha scritto: > On 2017-09-10 12:40, gerlando.fala...@gmail.com wrote: > >> > >> I suspect it's down to timing. > >> > >> What you're putting into the queue is a reference to the array, and it's > >> only some time later that the array itse

Re: array.array()'s memory shared with multiprocessing.Process()

2017-09-10 Thread MRAB
On 2017-09-10 12:40, gerlando.fala...@gmail.com wrote: I suspect it's down to timing. What you're putting into the queue is a reference to the array, and it's only some time later that the array itself is pickled and then sent (the work being done in the 'background'). Modifying the array b

Re: array.array()'s memory shared with multiprocessing.Process()

2017-09-10 Thread gerlando . falauto
> > I suspect it's down to timing. > > What you're putting into the queue is a reference to the array, and it's > only some time later that the array itself is pickled and then sent (the > work being done in the 'background'). > > Modifying the array before (or while) it's actually being sent

Re: array.array()'s memory shared with multiprocessing.Process()

2017-09-09 Thread MRAB
On 2017-09-09 22:23, iurly wrote: Hi, I'm writing a multiprocessing program whose behavior I don't understand. Essentially, the main process collects data and then passes it to a consumer process. For performance reasons I'm using a "static" circular buffer created through array.array(), and t

array.array()'s memory shared with multiprocessing.Process()

2017-09-09 Thread iurly
Hi, I'm writing a multiprocessing program whose behavior I don't understand. Essentially, the main process collects data and then passes it to a consumer process. For performance reasons I'm using a "static" circular buffer created through array.array(), and then passing it "as-is" by pushing it