Re: [Twisted-Python] Some things I've learned: safer callbacks, better t.p.context

[Glyph Lefkowitz]

> On Oct 18, 2016, at 7:09 PM, Kevin Conway  wrote:
> 
> > making such aggressive use of private APIs that it could win a contest 
> > about how to ensure that you break on every new release of Twisted :)
> 
> We're very aware of that! It's one of the reasons we have the test matrix set 
> up to run multiple versions of Python and Twisted. I have not started on 16.X 
> compatibility yet.
> 
>  >  I imagine that it's a non-trivial impact to performance, so it would be 
> worthwhile to track that.
> 
> We put this this through some extensive benchmarks and testing to measure the 
> performance impact. For example, the details are logged in a commit message 
> but, we initially implemented the @inlineCallbacks extension as a coroutine 
> wrapper. However, we found that the way t.p.Failure tries to serialize 
> itself, and its local+global scopes, to a dictionary caused enormous memory 
> and CPU consumption when triggered because of the added objects in those 
> spaces. The negative impact grew exponentially with levels of nested 
> coroutines. Very bad day.

What are you referring to as a "coroutine" here?  A generator?  And exponential 
growth, you say?  That sounds very surprising.

> Once we pivoted to a small fork of @inlineCallbacks, we measured the overall 
> performance hit to be negligible in our services. I'll dig around to see if I 
> can find where we documented the actual numbers we saw. At a macro level, our 
> service wide stats showed no meaningful growth of runtime or memory 
> consumption.

Does this mean you only get context tracking against inlineCallbacks, and not 
other usages of Deferred?

> > digression on "Don't Switch In The Core"
> 
> I was surprised at how much switching this context implementation was when we 
> put it in the lower level read/write callbacks. Each of our services process 
> a large amount of continually streaming data and our profiles show, IIRC, 
> that one of the top 5 consumers of CPU time was calling the read/write 
> callbacks. When we added this to those paths it increased overall CPU usage 
> by double digit percentage points. If this feature were available as an 
> opt-in reactor extension then providers could capacity plan around the 
> performance hit. We found it more valuable to move the switching closer to 
> application protocol code where switches happen less frequently.

Maybe "switching" is more expensive than I realized.  Where is this implemented?

>   > I also take it from the performance notes that you're not using PyPy?
> 
> We're still on cPython. PyPy is something we've talked about before but 
> haven't invested much time into yet. I don't know to what extent PyPy might 
> change the performance characteristics of the project.

As I always tell people - if you care about performance, PyPy should be step 
zero.  Optimizing for CPython looks like adding weird implementation-specific 
hacks that might start working or be backwards in the next version; optimizing 
for PyPy means making the code simpler and more readable so the JIT can figure 
out what to do ;).  So the pressure that optimizing for PyPy exerts on your 
code is generally a lot healthier.

-glyph___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Some things I've learned: safer callbacks, better t.p.context

[Itamar Turner-Trauring]

On Wed, Oct 19, 2016, at 05:45 PM, Itamar Turner-Trauring wrote:
> Well... I had a test that went through synchronous Deferred path. And
> yeah, it was easier to write than async test. But it failed to catch a
> bug that was only in async case. So the problem I see is that
> supporting both in Deferred means you need twice the number of tests
> each time you use Deferreds.
>

Er, that was unclear. I had a bug that wasn't caught by tests because it
passed with sync Deferred and failed with async Deferred callback, and I
didn't have tests for latter.

--
Itamar Turner-Trauring

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Some things I've learned: safer callbacks, better t.p.context

[Itamar Turner-Trauring]

On Tue, Oct 18, 2016, at 03:44 PM, Glyph Lefkowitz wrote:
>
> I think Deferred as it is today is a pretty good compromise between
> the two positions.  On the one hand it is decoupled from the event
> loop.  On the other - and this is important - *no Deferred-returning
> API will ever call your callbacks synchronously*.
> Deferred.addCallback will, of course, but savvy Twisted programmers
> can (and should) do this, if they have dependent state changes:
>
>> self.manipulateSomeStateForSetup()
>>
>> d = doSomethingPotentiallySynchronous()
>> *self.manipulateSomeStateForProcessing()*
>> d.addCallback(completeOperation)
>
> As a caller, you can always decide whether you can safely be re-
> entered or not.  In most cases, simply moving the 'addCallback' to the
> end of the function (a-la Go's "defer", oddly enough) is fine.  In
> more complex cases where you really need to unwind reentrancy
> completely, you can do your own callLater(0) or callFromThread() from
> an object with a reference to a reactor.

Well... I had a test that went through synchronous Deferred path. And
yeah, it was easier to write than async test. But it failed to catch a
bug that was only in async case. So the problem I see is that supporting
both in Deferred means you need twice the number of tests each time you
use Deferreds.

>> 3.
>
> What happened to '2'? :)

There were only two points :)
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Some things I've learned: safer callbacks, better t.p.context

[Glyph Lefkowitz]

> On Oct 19, 2016, at 2:47 PM, Itamar Turner-Trauring  
> wrote:
> 
> On Wed, Oct 19, 2016, at 05:45 PM, Itamar Turner-Trauring wrote:
>> Well... I had a test that went through synchronous Deferred path. And yeah, 
>> it was easier to write than async test. But it failed to catch a bug that 
>> was only in async case. So the problem I see is that supporting both in 
>> Deferred means you need twice the number of tests each time you use 
>> Deferreds.
> 
> Er, that was unclear. I had a bug that wasn't caught by tests because it 
> passed with sync Deferred and failed with async Deferred callback, and I 
> didn't have tests for latter.


To be clear: I do see this as a downside to Deferred's architecture; it's a 
tradeoff.  I see the loose coupling with the event loop as a worthwhile upside.

However, it's totally possible to write the async Deferred callback case as 
well, just by having the test fire the Deferred after returning to the test 
body instead of firing it before :-).

-glyph___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Some things I've learned: safer callbacks, better t.p.context

[Jean-Paul Calderone]

It's also possible to write a little bit of testing library code and get
both versions of the test nearly for free.  Unfortunately, I very rarely
see test suites written that way.  I think many people don't realize there
are two cases to handle or believe testing both cases is too expensive for
them (perhaps because they don't realize you can reuse the code for doing
so to save on the cost).

Jean-Paul

On Wed, Oct 19, 2016 at 5:54 PM, Glyph Lefkowitz 
wrote:

>
> On Oct 19, 2016, at 2:47 PM, Itamar Turner-Trauring 
> wrote:
>
> On Wed, Oct 19, 2016, at 05:45 PM, Itamar Turner-Trauring wrote:
>
> Well... I had a test that went through synchronous Deferred path. And
> yeah, it was easier to write than async test. But it failed to catch a bug
> that was only in async case. So the problem I see is that supporting both
> in Deferred means you need twice the number of tests each time you use
> Deferreds.
>
>
> Er, that was unclear. I had a bug that wasn't caught by tests because it
> passed with sync Deferred and failed with async Deferred callback, and I
> didn't have tests for latter.
>
>
> To be clear: I do see this as a *downside* to Deferred's architecture;
> it's a tradeoff.  I see the loose coupling with the event loop as a
> worthwhile upside.
>
> However, it's totally possible to write the async Deferred callback case
> as well, just by having the test fire the Deferred after returning to the
> test body instead of firing it before :-).
>
> -glyph
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Some things I've learned: safer callbacks, better t.p.context

[Amber "Hawkie" Brown]

> On 20 Oct. 2016, at 08:54, Glyph Lefkowitz  wrote:
> 
> 
>> On Oct 19, 2016, at 2:47 PM, Itamar Turner-Trauring > > wrote:
>> 
>> On Wed, Oct 19, 2016, at 05:45 PM, Itamar Turner-Trauring wrote:
>>> Well... I had a test that went through synchronous Deferred path. And yeah, 
>>> it was easier to write than async test. But it failed to catch a bug that 
>>> was only in async case. So the problem I see is that supporting both in 
>>> Deferred means you need twice the number of tests each time you use 
>>> Deferreds.
>> 
>> Er, that was unclear. I had a bug that wasn't caught by tests because it 
>> passed with sync Deferred and failed with async Deferred callback, and I 
>> didn't have tests for latter.
> 
> 
> To be clear: I do see this as a downside to Deferred's architecture; it's a 
> tradeoff.  I see the loose coupling with the event loop as a worthwhile 
> upside.

One thing that I have still not figured out is how Futures (which are tightly 
tied to an event loop) will possibly ever work over multiple event loops. I 
think since you can't really chain them, this is less of a problem, but 
Deferreds like to absorb other Deferreds and make one big one, which would make 
the tight coupling problematic if you wanted to use two event loops (like, say, 
a GTK one and an IOCP one on Windows).

- Amber

> 
> However, it's totally possible to write the async Deferred callback case as 
> well, just by having the test fire the Deferred after returning to the test 
> body instead of firing it before :-).
> 
> -glyph
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Some things I've learned: safer callbacks, better t.p.context

[Glyph Lefkowitz]

> On Oct 19, 2016, at 4:25 PM, Jean-Paul Calderone  
> wrote:
> 
> It's also possible to write a little bit of testing library code and get both 
> versions of the test nearly for free.  Unfortunately, I very rarely see test 
> suites written that way.  I think many people don't realize there are two 
> cases to handle or believe testing both cases is too expensive for them 
> (perhaps because they don't realize you can reuse the code for doing so to 
> save on the cost).

I don't think even I've ever written a test suite that way.  Could we provide 
any utilities in Twisted to make this easier, or even more importantly, more 
obvious?

-glyph___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Some things I've learned: safer callbacks, better t.p.context

[Glyph Lefkowitz]

> On Oct 19, 2016, at 4:43 PM, Amber Hawkie Brown  
> wrote:
> 
> One thing that I have still not figured out is how Futures (which are tightly 
> tied to an event loop) will possibly ever work over multiple event loops. I 
> think since you can't really chain them, this is less of a problem, but 
> Deferreds like to absorb other Deferreds and make one big one, which would 
> make the tight coupling problematic if you wanted to use two event loops 
> (like, say, a GTK one and an IOCP one on Windows).
> 

I've never seen this as a problem, because "multiple event loops" implicitly 
means "multiple threads"; and you really kinda need to call back a Deferred on 
the same event loop where it was originated.  When would you want to do this?

-glyph___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Some things I've learned: safer callbacks, better t.p.context

[Kevin Conway]

> What are you referring to as a "coroutine" here?  A generator?

Generator, yes, but used as a coroutine that both emits values and receives
them. The introduction of dedicated coroutines in Python 3.5 is something I
haven't updated my personal language for very well. Generator-coroutine is
the term used in there Python docs, IIRC.

> And exponential growth, you say?

Well... That may not be perfectly accurate and I shouldn't have used that
term. It grew large enough, fast enough that the same level of nesting was
measured in microseconds for the control and and over an hour for the
wrapper so I cancelled the test. Let me put together a reproducible
experiment and let others decide the growth rate.

> Where is this implemented?

Unfortunately, our production use of the lib is not something we've open
sourced. Similar to the above, I'll make available a reproducible
experiment so you can draw your own conclusion.

> Optimizing for CPython looks like adding weird implementation-specific
hacks that might start working or be backwards in the next version

This went into the rationale behind making most of the components
pluggable. The choice of optimizing for cPython is internal to us and we
chose to document it for others. It's not required and not the default.

As far as the possibility of integrating this feature into Twisted, let me
chat with my cohorts over here. Well come back with some benchmark code and
a proposal of changes for further discussion. I'll probably start a new
thread to focus on the context bits.

On Wed, Oct 19, 2016, 16:56 Glyph Lefkowitz  wrote:


On Oct 19, 2016, at 2:47 PM, Itamar Turner-Trauring 
wrote:

On Wed, Oct 19, 2016, at 05:45 PM, Itamar Turner-Trauring wrote:

Well... I had a test that went through synchronous Deferred path. And yeah,
it was easier to write than async test. But it failed to catch a bug that
was only in async case. So the problem I see is that supporting both in
Deferred means you need twice the number of tests each time you use
Deferreds.


Er, that was unclear. I had a bug that wasn't caught by tests because it
passed with sync Deferred and failed with async Deferred callback, and I
didn't have tests for latter.


To be clear: I do see this as a *downside* to Deferred's architecture; it's
a tradeoff.  I see the loose coupling with the event loop as a worthwhile
upside.

However, it's totally possible to write the async Deferred callback case as
well, just by having the test fire the Deferred after returning to the test
body instead of firing it before :-).

-glyph
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python