I've got a bit of a problem with copy.deepcopy and using
multiprocessing.Queue.
I have an HTTPAPI that gets exposed to add objects to a
multiprocessing.Qeue. Source code here:
https://github.com/jmdevince/cifpy3/blob/master/lib/cif/api/handler.py#L283
The trouble is, even using deepcopy, m
I've got a bit of a problem with copy.deepcopy and using
multiprocessing.Queue.
I have an HTTPAPI that gets exposed to add objects to a
multiprocessing.Qeue. Source code here:
https://github.com/jmdevince/cifpy3/blob/master/lib/cif/api/handler.py#L283
The trouble is, even using deepcopy, m
Anyone have any ideas? I feel like this could be a bug with the garbage
collector across multiprocessing.
From: James DeVincentis [mailto:ad...@hexhost.net]
Sent: Wednesday, October 14, 2015 12:41 PM
To: 'python-list@python.org'
Subject: Problem with copy.deepcopy and multiproces
to
a multiprocess.Queue it fixed the issue.
Kind of odd to me. Not sure if anyone wants to look into it.
> On Oct 15, 2015, at 5:42 PM, Ian Kelly wrote:
>
> On Thu, Oct 15, 2015 at 4:02 PM, James DeVincentis wrote:
> >
> > Anyone have any ideas? I feel like this could be a
I take that back. It’s not entirely fixed.
Something else strange is going on here. More debugging needed.
> On Oct 15, 2015, at 6:36 PM, James DeVincentis wrote:
>
> I think I tracked this down and resolved it.
>
> It appears taking an object from a multiprocess.Queue and
Size: 572
> On Oct 15, 2015, at 5:02 PM, James DeVincentis wrote:
>
> Anyone have any ideas? I feel like this could be a bug with the garbage
> collector across multiprocessing.
>
> From: James DeVincentis [mailto:ad...@hexhost.net]
> Sent: Wednesday, October 14, 2015 1
ing multiprocessing.Queue._feed().
Thoughts from anyone?
> On Oct 15, 2015, at 7:37 PM, James DeVincentis wrote:
>
> Looking into it, I seem to have found a race condition where a
> multiprocessing.Queue.get() can get hung waiting for an object even if there
> are objects in th
is why is it necessary to call os._exit() ?
> On Oct 17, 2015, at 3:19 PM, James DeVincentis wrote:
>
> So, whatever is causing this is a bit deeper in the multiprocessing.Queue
> class. I tried using a non-blocking multiprocessing.Queue.get() by setting
> the first parameter to
to exit nicely (not raising a SystemExit), AND flush
the buffers, so I’m stuck in a stupid edge case.
> On Oct 18, 2015, at 3:33 AM, James DeVincentis wrote:
>
> Seems I found the cause. os._exit() is used in ForkingMixIn for SocketServer
> and it’s child classes.
>
> Since
I see, looks like I’ll have to use Queue.close()
Didn’t think it would be necessary since I was assuming it would be garbage
collected. Sigh. Bug, fixed.
Thanks everyone!
> On Oct 18, 2015, at 3:41 AM, James DeVincentis wrote:
>
> I get why it needs to be called, but this looks like
015, at 3:19 PM, James DeVincentis wrote:
>
> So, whatever is causing this is a bit deeper in the multiprocessing.Queue
> class. I tried using a non-blocking multiprocessing.Queue.get() by setting
> the first parameter to false and then catching the queue.Empty exception. For
>
18, 2015, at 10:55 AM, Ian Kelly wrote:
>
> On Sun, Oct 18, 2015 at 2:46 AM, James DeVincentis <mailto:ad...@hexhost.net>> wrote:
> >
> > I see, looks like I’ll have to use Queue.close()
> >
> > Didn’t think it would be necessary since I was assuming it
12 matches
Mail list logo
