Re: Agent as a processing queue

2009-01-25 Thread Timothy Pratley
On Jan 23, 10:43 pm, e wrote: > Just to understand ... 'send-future' works too but you don't think that's > the way to go? The Agent send pool has a limited size, so doing recursive calls with a stack greater than that size will result in a freeze (waiting for new threads that can't be made ye

Re: Agent as a processing queue

2009-01-24 Thread e
> Now you > wont be able to run this unless you modify Agent.java, search for > soloExectuor, make it public, and recompile Clojure. is this a code change people want to make? > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to

Re: Agent as a processing queue

2009-01-23 Thread e
Thanks. I really appreciate seeing that example. There are more answers there than even the original question . . . like how easily you time the performance and how easily you make random lists of values. . . .also the use of "remove" in that code you grabbed to avoid having two filters is totall

Re: Agent as a processing queue

2009-01-23 Thread Timothy Pratley
On Jan 23, 11:24 am, e wrote: > wow.  I wonder if I could use this for the quicksort I was talking about. Interesting question. I can't think of an elegant way to implement an agent based parallel quicksort because await cannot be used recursively. So I implemented a version using Futures inst

Re: Agent as a processing queue

2009-01-22 Thread Mark H.
On Jan 22, 4:24 pm, e wrote: > At first, there will only be one thread for the first pass, but the number > will grow as subtasks fire sub-sub tasks.  I am sure this is a silly way to > sort because the threadspawning time is so much longer than the filtering > time, but it's just an exercise. I

Re: Agent as a processing queue

2009-01-22 Thread e
wow. I wonder if I could use this for the quicksort I was talking about. I would need to have the function being added as a job be able to add it's own jobs recursively . . . .and kill jobs when they are done. But do you have to sleep, or is there a way to join the children when they are done?

Re: Agent as a processing queue

2009-01-22 Thread Emeka
Tim, Could you explain atoms the way you explained agents?\ Emeka --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from th

Re: Agent as a processing queue

2009-01-22 Thread Timothy Pratley
> (both CPU and IO bound at different stages of processing) so > it's ideal to have a thread pool to process different tasks in > parallel, even though they are independent. If you use the agents, the underlying implementation uses two thread pools: (1) static relative to your processors, use "se

Re: Agent as a processing queue

2009-01-22 Thread Christian Vest Hansen
On Thu, Jan 22, 2009 at 2:52 PM, Greg Harman wrote: > > I like the example, and I should be able to adapt it. I do have one > question about the example (probably not related to agents: > > ; queue up some work to be done > (defn add-job [func] > (let [a (agent nil)] >(send a (fn [_] (func))

Re: Agent as a processing queue

2009-01-22 Thread Greg Harman
I like the example, and I should be able to adapt it. I do have one question about the example (probably not related to agents: ; queue up some work to be done (defn add-job [func] (let [a (agent nil)] (send a (fn [_] (func))) (swap! jobs #(conj %1 a What is the notation fn [_]? I

Re: Agent as a processing queue

2009-01-22 Thread Greg Harman
Thanks Tim, I'll have a look at that. To clarify my use case, I was thinking of events that can be processed sequentially but that may take a non-trivial amount of time to complete (both CPU and IO bound at different stages of processing) so it's ideal to have a thread pool to process different t

Re: Agent as a processing queue

2009-01-21 Thread Timothy Pratley
Hi Greg Here is a proof of concept of one approach you could take: http://groups.google.com/group/clojure/web/job-queue.clj A set of agents are maintained to represent computation jobs. When the results are gathered, the agent is thrown away. I think using multiple agents in this way could be qu

Agent as a processing queue

2009-01-21 Thread Greg Harman
I'd like to implement a processing queue in which I'll asynchronously drop events (represented by, say, a structmap) onto a queue, and retrieve the results later. One possible approach to this could be using an agent to store the results (the agent's controlled state is a collection of results fr