This question about forced data consumption has come up before: https://github.com/apache/httpcomponents-core/pull/91#issuecomment-432868169
On Sun, Sep 8, 2019 at 5:08 PM Roy Hashimoto <roy.hashim...@gmail.com> wrote: > By looking at the suggested 5.0 examples I was able to get an > AsyncServerExchangeHandler subclass to play nicely with Kotlin coroutines > on the AsyncDataProducer side of things, i.e. minimizing produce() polling > and avoiding buffering. > > I haven't been as successful with throttling calls on the AsyncDataConsumer > side, i.e. consume() calls keep being made even though the capacity window > has gone negative. I think this might be the expected behavior because of > this comment in AbstractHttp1StreamDuplexer: > > // At present the consumer can be forced to consume data > // over its declared capacity in order to avoid having > // unprocessed message body content stuck in the session > // input buffer > > Does that refer to just the case where the capacity starts positive but > data exceeding the capacity is delivered to consume()? Or does it refer to > the behavior I see, which is that capacity updates (or the lack of them) > don't seem to have any effect for HTTP/1.1? > > I've also tried running the Http1IntegrationTest.testSlowResponseConsumer() > test, substituting this line to trigger updateCapacity() calls: > > > > client.start(Http1Config.custom().setBufferSize(256).setInitialWindowSize(32).build()); > > By adding the small initial window, I can see the capacityWindow going more > and more negative on each consume() call with all the data buffered before > the test code completes its first sleep. > > I don't have a specific use case for a slow consumer, just want to know if > I'm misunderstanding something. > > Thanks! > Roy > > On Fri, Sep 6, 2019 at 10:15 AM Roy Hashimoto <roy.hashim...@gmail.com> > wrote: > > > Those are good leads, I'll pursue them. > > > > Thanks! > > Roy > > > > On Fri, Sep 6, 2019 at 9:57 AM Oleg Kalnichevski <ol...@apache.org> > wrote: > > > >> On Fri, 2019-09-06 at 09:43 -0700, Ryan Schmitt wrote: > >> > Have you looked at the reactive extensions for HttpCore5? They > >> > demonstrate > >> > how to implement AsyncEntityProducer/AsyncDataProducer with support > >> > for > >> > backpressure (or you can just use the Reactive Streams API instead): > >> > > >> > > >> > >> > https://github.com/apache/httpcomponents-core/tree/master/httpcore5-reactive/src/main/java/org/apache/hc/core5/reactive > >> > > >> > > >> > >> Just a bit of background. In 5.0 one can no longer assume that one > >> message exchange has exclusive ownership of the underlying connection. > >> Multiplexed message exchanges in HTTP/2 and piplelined message > >> exchanges in HTTP/1.1 must not block other concurrent exchanges. > >> Message changes however can update their current capacity via > >> `CapacityChannel`. Reactive extensions is a great example and also an > >> alternative to the native APIs per Ryan's recommendation. > >> > >> If you prefer the native APIs you can take a look at the classic I/O > >> adaptors that essentially emulate the classic blocking i/o on top of > >> the new async APIs [1] or HTTP/1.1 integration tests [2] that have a > >> number of 'slow' consumer / producer test cases. > >> > >> Cheers > >> > >> Oleg > >> > >> [1] > >> > https://github.com/apache/httpcomponents-core/tree/master/httpcore5/src/main/java/org/apache/hc/core5/http/nio/support/classic > >> [2] > >> > https://github.com/apache/httpcomponents-core/blob/master/httpcore5-testing/src/test/java/org/apache/hc/core5/testing/nio/Http1IntegrationTest.java > >> > >> > >> > On Fri, Sep 6, 2019 at 9:33 AM Roy Hashimoto <roy.hashim...@gmail.com > >> > > > >> > wrote: > >> > > >> > > I'm playing with asynchronous handlers in HttpCore 5, and I'd like > >> > > to have > >> > > an AsyncEntityProducer write data at its own (slow) rate like in > >> > > this old > >> > > thread < > >> > > https://marc.info/?l=httpclient-commons-dev&m=134928851229305&w=2 > >> > > > . > >> > > > >> > > Writing to the DataStreamChannel whenever I want - outside the > >> > > scope of a > >> > > produce() method call - works fine, but I notice that produce() is > >> > > being > >> > > called every 5-6 milliseconds which ideally I would like to > >> > > eliminate or > >> > > reduce. > >> > > > >> > > The answer in the old thread was to use IOControl.suspendOutput() > >> > > and > >> > > IOControl.requestOutput(), but this class appears no longer to be > >> > > in > >> > > HttpCore 5. I see that there is a DataStreamChannel.requestOutput() > >> > > but I > >> > > haven't figured out what suspension call that should be paired > >> > > with. I have > >> > > tried simply returning 0 from my AsyncEntityProducer.available() > >> > > override, > >> > > but that doesn't seem to be it. > >> > > > >> > > Is there a new way to suspend/resume output in HttpCore 5? > >> > > > >> > > Thanks! > >> > > Roy > >> > > > >> > > Kotlin source here > >> > > < > >> > > https://gist.github.com/rhashimoto/1f5501d3b5d2aa95251fe12f4f0be250 > >> > > >. > >> > > > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org > >> For additional commands, e-mail: dev-h...@hc.apache.org > >> > >> >
