Hello Paul, Why not just adjust the number of workers based on actual workload instead of monitoring the source channel for average depth ?
Have a look at this code<https://github.com/halgari/clojure-conj-2013-core.async-examples/blob/master/src/clojure_conj_talk/core.clj#L254-287>from clojure-conj presentation about core.async, it's just a simple thread-pool service, where you can specify source-channel, maximal number of threads (workers), function to be performed by each worker and timeout. Timeout means, for how long each thread will be waiting if there is no data on input before it will shut-down. The service starts with minimal amount of threads able to handle the workload and then progressively scales up to max specified number of threads if there are no "free" threads able to take items from the source channel. Maybe i misunderstood your description of the problem, but for me, it seems like a perfect fit. Jan Dňa utorok, 21. januára 2014 17:56:56 UTC+1 Paul Viola napísal(-a): > > I think this is all well and good for a particular use of channel. > > So perhaps I am misusing channels?? > > To repeat: in one case I have workers pulling from a channel of *real > work*. For various reasons this channel might get filled rather deeply. > In this case I would want to add additional workers or get a bigger > machine. I was wondering if monitoring the channel for things like average > depth (or 99 percentile) would give me the information I needed. > > I could of course "just skip the channel business, and use a java queue" > is a fine proposal. > > But since the producers of this work are truly asynchronous (attached to > the real world) I thought it best to keep the channel methodology. > > > > > > > On Tue, Jan 21, 2014 at 5:11 AM, Aaron France > <aaron.l...@gmail.com<javascript:> > > wrote: > >> On 21/01/14 14:09, Moritz Ulrich wrote: >> >>> On Tue, Jan 21, 2014 at 9:43 AM, Aaron France >>> <aaron.l...@gmail.com<javascript:>> >>> wrote: >>> >>>> Since channels yield nil when they are devoid of items, surely this is >>>> enough to know when the channel is empty? >>>> >>> That's not correct. Take-Operations block on empty channels. They >>> yield nil when they're closed. You could add a timeout to the take >>> operation to see if no item arrived in a specific time. >>> >>> Much appreciated for the clarification. It's the same in Go. >> >> I can imagine this pattern (take on a possibly closed channel being >> useful) being useful but I'm not convinced knowing the count of channel is >> a safe thing to know/care about. >> >> My $0.02, perhaps Clojure does this differently. >> >> >> -- >> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clo...@googlegroups.com<javascript:> >> Note that posts from new members are moderated - please be patient with >> your first post. >> To unsubscribe from this group, send email to >> clojure+u...@googlegroups.com <javascript:> >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> --- You received this message because you are subscribed to a topic in >> the Google Groups "Clojure" group. >> To unsubscribe from this topic, visit https://groups.google.com/d/ >> topic/clojure/zD2jl-bIFXI/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> clojure+u...@googlegroups.com <javascript:>. >> For more options, visit https://groups.google.com/groups/opt_out. >> > > -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
