Re: [Twisted-Python] finding out what's blocking the reactor

2019-05-15 Thread Werner Thie
Aloha the code below has proven to be a life saver together with manhole. It might be a bit outdated but I usually create a manhole access allowing me to inspect the server from within then using dumpObjects to look at the sea of objects in the running process. Even only watching the inner workin

Re: [Twisted-Python] finding out what's blocking the reactor

2019-05-15 Thread Maarten ter Huurne
On Wednesday, 15 May 2019 13:11:29 CEST Scott, Barry wrote: > In my experience gc is low level noise only. > Our work load is very heavy and its never been an issue. It depends entirely on how many objects you have. With a few GBs worth of relatively small objects, a full garbage collection cycl

Re: [Twisted-Python] finding out what's blocking the reactor

2019-05-15 Thread Scott, Barry
On Wednesday, 15 May 2019 06:30:54 BST Chris Withers wrote: > On 14/05/2019 22:14, Werner Thie wrote: > > Hi all > > > > with periods of seemingly no activity the gc comes to mind. > > Hmm, gc blocking the process is an interesting thought, what evidence > would you see with gdb, etc, to show tha

Re: [Twisted-Python] finding out what's blocking the reactor

2019-05-15 Thread Scott, Barry
On Wednesday, 15 May 2019 06:27:30 BST Chris Withers wrote: > On 14/05/2019 16:39, Scott, Barry wrote: > > On Monday, 13 May 2019 15:26:01 BST Chris Withers wrote: > >> Hi All, > >> > >> On some busy instances of the app I'm working with/on, we see websocket > >> connections being dropped due to n

Re: [Twisted-Python] finding out what's blocking the reactor

2019-05-15 Thread Maarten ter Huurne
On Wednesday, 15 May 2019 07:30:54 CEST Chris Withers wrote: > On 14/05/2019 22:14, Werner Thie wrote: > > Hi all > > > > with periods of seemingly no activity the gc comes to mind. > > Hmm, gc blocking the process is an interesting thought, what evidence > would you see with gdb, etc, to show th

Re: [Twisted-Python] finding out what's blocking the reactor

2019-05-15 Thread Chris Withers
On 14/05/2019 16:39, Scott, Barry wrote: On Monday, 13 May 2019 15:26:01 BST Chris Withers wrote: Hi All, On some busy instances of the app I'm working with/on, we see websocket connections being dropped due to not ping/ponging in time, I'm fairly sure this is as a result of some user-written c

Re: [Twisted-Python] finding out what's blocking the reactor

2019-05-15 Thread Chris Withers
On 14/05/2019 22:14, Werner Thie wrote: Hi all with periods of seemingly no activity the gc comes to mind. Hmm, gc blocking the process is an interesting thought, what evidence would you see with gdb, etc, to show that this was the problem? Chris ___

Re: [Twisted-Python] finding out what's blocking the reactor

2019-05-14 Thread Glyph
> On May 14, 2019, at 8:39 AM, Scott, Barry wrote: > > On Monday, 13 May 2019 15:26:01 BST Chris Withers wrote: >> Hi All, >> >> On some busy instances of the app I'm working with/on, we see websocket >> connections being dropped due to not ping/ponging in time, I'm fairly >> sure this is as a

Re: [Twisted-Python] finding out what's blocking the reactor

2019-05-14 Thread Werner Thie
Hi all with periods of seemingly no activity the gc comes to mind. Is the working set size stable or ever increasing, swap operations might block activity of the process I had problems in the beginning with twisted, because I was holding on to a lot of objects and thus creating  circular non col

Re: [Twisted-Python] finding out what's blocking the reactor

2019-05-14 Thread Scott, Barry
On Monday, 13 May 2019 15:26:01 BST Chris Withers wrote: > Hi All, > > On some busy instances of the app I'm working with/on, we see websocket > connections being dropped due to not ping/ponging in time, I'm fairly > sure this is as a result of some user-written callbacks blocking the > reactor fo

Re: [Twisted-Python] finding out what's blocking the reactor

2019-05-13 Thread Glyph
> On May 13, 2019, at 7:26 AM, Chris Withers wrote: > > Hi All, > > On some busy instances of the app I'm working with/on, we see websocket > connections being dropped due to not ping/ponging in time, I'm fairly sure > this is as a result of some user-written callbacks blocking the reactor f