We don't have streams of data here, the long running tasks have side-effects. I would prefer to avoid adding another whole framework just to run a few long running jobs in p//.
I have a list of jobs to do, I'm partitioning that list up into 4 sub lists to be worked through by 4 p// workers, I then want to block and wait until all 4 workers have finished their tasks. On Wednesday, September 17, 2014 3:27:07 AM UTC+10, larry google groups wrote: > > > This does not look correct to me. Perhaps someone else has more insight > into this. I am suspicious about 2 things: > > 1.) your use of doall > > 2.) your use of (thread) > > It looks to me like you are trying to hack together a kind of pipeline or > channel. Clojure has a wealth of libraries that can handle that for you. > The main thing you are trying to do is this: > > (long-running-widget-processor widget)) > > > You go to some trouble to set up workers, all to ensure that > long-running-widget-processor > is handled in its own thread. > > I would suggest you look at Lamina: > > https://github.com/ztellman/lamina > > In particular, look at pipelines: > > https://github.com/ztellman/lamina/wiki/Pipelines > > > > > > On Friday, September 5, 2014 1:46:02 AM UTC-4, Beau Fabry wrote: >> >> Is the kinda ugly constant (doall usage a sign that I'm doing something >> silly? >> >> (let [num-workers 4 >> widgets-per-worker (inc (int (/ (count widgets) num-workers))) >> bucketed-widgets (partition-all widgets-per-worker widgets) >> workers (doall (map (fn [widgets] >> (thread >> (doseq [widget widgets] >> (long-running-widget-processor widget)) >> true)) >> bucketed-widgets))] >> (doall (map <!! workers))) >> >> https://gist.github.com/bfabry/ad830b1888e4fc550f88 >> >> All comments appreciated :-) >> >> Cheers, >> Beau >> > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
