Sorry, didn't cover converse case. That's handled by the ChannelSink directly underneath. Note that each write returns a Netty ChannelFuture representing completion of the write, which is transformed into a Manifold deferred. Any time a Manifold put returns an unrealized deferred, that creates upstream backpressure.
On Wed, Oct 8, 2014 at 10:16 AM, Zach Tellman <ztell...@gmail.com> wrote: > The documentation for Manifold can explain the API better than I can > here. The point where that interacts with Netty w.r.t. backpressure is > here: > https://github.com/ztellman/aleph/blob/0.4.0/src/aleph/netty.clj#L109. > Here the stream represents data coming off the wire, and if the put onto > the stream is not immediately successful, backpressure is enabled until the > put completes. No blocking required anywhere. > > On Wed, Oct 8, 2014 at 10:10 AM, Jozef Wagner <jozef.wag...@gmail.com> > wrote: > >> If you want to handle multiple TCP connections and async channels in one >> thread, you need a way how to block on both connections (wait for new input >> to arrive) and channels (wait for a free space in a buffer). Blocking only >> on connections will get you a busy loop if channels are full. If you could >> point me to the part of Aleph sources that handles this issue, I would be >> very grateful. I'm not familiar with netty API nor manifold's concepts, so >> I'm having trouble navigating in the Aleph sources. >> >> Thanks, >> Jozef >> >> On Wednesday, October 8, 2014 6:15:47 PM UTC+2, Zach Tellman wrote: >>> >>> I wasn't aware of hermod, that's interesting. I would still >>> characterize its approach to backpressure as "broken", though, since when >>> the queues get full it silently drops messages on the ground. In fairness, >>> this is very clearly documented, so it's less pernicious than some of the >>> other cases out there. >>> >>> Both core.async buffers and SelectableChannels have very particular >>> semantics, and I would be very surprised if they could be combined in that >>> way. It's perfectly possible to feed one into the other and handle >>> backpressure properly (again, I'm doing just that with Aleph 0.4.0, using >>> Netty), but it's a nuanced integration and easy to get wrong. >>> >>> On Wed, Oct 8, 2014 at 7:12 AM, <adrian...@mail.yu.edu> wrote: >>> >>>> Check out https://github.com/halgari/com.tbaldridge.hermod for an >>>> interesting take on this. >>>> >>>> On Wednesday, October 8, 2014 1:17:11 AM UTC-4, Sun Ning wrote: >>>>> >>>>> BTW, is there any network based core.async channel available now? >>>>> >>>>> On 10/08/2014 04:36 AM, adrian...@mail.yu.edu wrote: >>>>> >>>>> It's not about 'safety' (depending on what that means in this >>>>> context), but as Zach pointed out, if you aren't careful about >>>>> backpressure >>>>> you can run into performance bottlenecks with unrestrained async IO >>>>> operations because although they let you code as if you could handle an >>>>> unlimited amount of connections, obviously that isn't true. There is only >>>>> a >>>>> finite amount of data that can be buffered in and out of any network >>>>> according to its hardware. When you don't regulate that, your system will >>>>> end up spending an inordinate amount of time compensating for this. You >>>>> don't need to worry about this with "regular io" because the "thread per >>>>> connection" abstraction effectively bounds your activity within the >>>>> acceptable physical constraints of the server. >>>>> >>>>> On Tuesday, October 7, 2014 2:49:30 PM UTC-4, Brian Guthrie wrote: >>>>>> >>>>>> >>>>>> On Mon, Oct 6, 2014 at 12:10 AM, <adrian...@mail.yu.edu> wrote: >>>>>> >>>>>>> Zach makes an excellent point; I've used AsyncSocketChannels and its >>>>>>> irk (http://docs.oracle.com/javase/8/docs/api/java/nio/channels/ >>>>>>> AsynchronousServerSocketChannel.html), with core.async in the past. >>>>>>> Perhaps replacing your direct java.net.Sockets with nio classes that >>>>>>> can be >>>>>>> given CompletionHandlers (http://docs.oracle.com/javase >>>>>>> /7/docs/api/java/nio/channels/CompletionHandler.html) would be a >>>>>>> better fit. >>>>>>> >>>>>> >>>>>> Once I do some performance instrumentation I'll give that a shot. I >>>>>> admit that I'm not familiar with all the implications of using the nio >>>>>> classes; were I to switch, is it safe to continue using go blocks, or is >>>>>> it >>>>>> worth explicitly allocating a single thread per socket? >>>>>> >>>>>> Brian >>>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Clojure" group. >>>>> To post to this group, send email to clo...@googlegroups.com >>>>> Note that posts from new members are moderated - please be patient >>>>> with your first post. >>>>> To unsubscribe from this group, send email to >>>>> clojure+u...@googlegroups.com >>>>> For more options, visit this group at >>>>> http://groups.google.com/group/clojure?hl=en >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Clojure" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to clojure+u...@googlegroups.com. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Clojure" group. >>>> To post to this group, send email to clo...@googlegroups.com >>>> Note that posts from new members are moderated - please be patient with >>>> your first post. >>>> To unsubscribe from this group, send email to >>>> clojure+u...@googlegroups.com >>>> For more options, visit this group at >>>> http://groups.google.com/group/clojure?hl=en >>>> --- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "Clojure" group. >>>> To unsubscribe from this topic, visit https://groups.google.com/d/ >>>> topic/clojure/TVMQJwaij1U/unsubscribe. >>>> To unsubscribe from this group and all its topics, send an email to >>>> clojure+u...@googlegroups.com. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clojure@googlegroups.com >> Note that posts from new members are moderated - please be patient with >> your first post. >> To unsubscribe from this group, send email to >> clojure+unsubscr...@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> --- >> You received this message because you are subscribed to a topic in the >> Google Groups "Clojure" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/clojure/TVMQJwaij1U/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> clojure+unsubscr...@googlegroups.com. >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
