The Java executor classes are essentially a queue backed by a thread pool of workers.
So suppose we create a new executor: (def executor (java.util.concurrent.Executors/newFixedThreadPool 32)) This creates an ExecutorService object with a pool of 32 worker threads. We can pass it a zero-argument function to execute: (.execute executor (fn [] (prn "Hello World"))) This function is placed on a queue until one of the 32 worker threads is available to process it. In this case the function prints a message, but you could write something to send an email instead: (defn queue-email [email] (.execute executor (fn [] (send-email email))) If we execute "queue-email" 10,000 times, then 10,000 "send-email" jobs will be passed to the executor. The executor will pass them onto its 32 workers, effectively processing 32 at a time until it exhausts the queue. If you want fewer emails to be sent at once, you can reduce the worker thread pool size. In my earlier example, I used an executor with only one worker thread. On Thu, 25 Oct 2018 at 18:17, <brj...@gmail.com> wrote: > Hi James, > > Thanks! How would one create a thread that continuously monitors a mail > queue and sends any mails in it? Or do you mean that I would create one > thread per mail that I want to send? Wouldn't that create a problem if I'm > about to send 10000 mails in one go? > > And thanks for the heads up regarding indefinite blocking - I'll make sure > to guard against that. > > Thanks, > Brjánn > > On Thu, 25 Oct 2018 at 18:57, James Reeves <ja...@booleanknot.com> wrote: > >> Hi Brjánn, >> >> Executing queued jobs in a background thread is a common task, and as it >> happens there's a set of Java classes to do just that. >> >> (let [e (java.util.concurrent.Executors/newSingleThreadExecutor)] >> (.execute e #(prn "foo")) >> (.execute e #(prn "bar")) >> (Thread/sleep 1000) >> (.execute e #(prn "baz"))) >> >> Don't worry about cleaning up a single thread. An inactive thread takes >> up a minimal amount of VM memory and no CPU. You're going to use up more OS >> resources deleting and recreating the thread. >> >> You do need to be careful that sending an email doesn't block >> indefinitely, but usually I/O operations come with timeouts. Just ensure >> you set a reasonable timeout and do something sensible when you can't send. >> >> On Thu, 25 Oct 2018 at 16:46, <brj...@gmail.com> wrote: >> >>> Hi, >>> >>> First, I would like to briefly present myself to the list. I'm a >>> psychologist and researcher at the Karolinska Institutet in Stockholm, >>> Sweden. I do research within the field of internet-based psychological >>> treatment - which is a field that has grown a lot during the last 10-15 >>> years. I have developed my own web platform to deliver these treatments to >>> patients with different mental and medical conditions. I wrote the first >>> version in PHP, and not being a professional programmer, the code base is a >>> big mess after > 10 years of development. >>> >>> Two years ago, a friend introduced me to Clojure. I had never worked >>> with a lisp before, but I quickly fell completely in love with it. So many >>> of the issues of programming in PHP completely disappeared! Words cannot >>> express my gratitude to Rich for inventing Clojure, the core team, all >>> developers out there who write extremely useful libraries, and my friend >>> for introducing Clojure to me. >>> >>> I decided to completely rewrite my web platform in Clojure. I've chosen >>> to do a gradual move from PHP to Clojure, meaning that I replace PHP code >>> with Clojure code component by component and continuously test in >>> production. Like I said, I'm not a professional programmer, and this >>> venture poses a lot of challenges. I try to read everything I find in books >>> and on the web, but when it comes to more complex issues, such as threads >>> and async programming, I feel that I end up almost guessing and with a lot >>> of trial and error. I have no idea how cautious one should be when >>> launching a new thread (while it completely occupy the server???) and am >>> often surprised when my go blocks suddenly freeze. I feel that I am at the >>> mercy of the Clojure community if I want to understand these (and many >>> other!) issues. >>> >>> This leads me to the subject of this email. I've decided to migrate my >>> mail queue from PHP to Clojure. In PHP, it's just a cron job that executes >>> every five minutes to send all emails (and actually also SMS-messages, but >>> not really relevant) that have been queued. >>> >>> I've written a basic mock Clojure implementation with the following goals >>> - All messages should be sent async, i..e, the web user should not have >>> to wait while the email is being sent. -> I'm sending them in a separate >>> thread. >>> - I have a fear that if I have a thread dedicated only to sending >>> emails, I'm wasting server resources. -> I want the thread to die 5 seconds >>> after the last email in the queue has been sent. >>> >>> My implementation basically consists of >>> - An eternal go loop that receives a notification through a channel if >>> new messages have been queued >>> - The go loop checks if the mail sender thread is running. If not, it >>> starts it. >>> - The mail sender thread dies 5 secs after the last email was sent >>> - The state of the thread (running / not running) is stored in an agent >>> to avoid race conditions (i.e., starting multiple threads or not starting a >>> thread because it is running when its status is checked but stops right >>> after). >>> >>> My code is here >>> https://gist.github.com/brjann/2aef16849b9bd445374cb6b31efece60 >>> >>> If any of you have had the time and energy to read this far (including >>> the code), I would be very grateful for your input. >>> - Is there a risk that my go block will hang? >>> - Have I eliminated the risk for race conditions? >>> - Do I really need to kill the thread or is there no risk for thread >>> starvation on the server (I will probably >>> - Could I use send instead of send-off? I guess that I am now using two >>> threads, one for the sender and one each time I send a message using >>> send-off. >>> - Any newbie mistakes / coding style issues? >>> - Could this be done in a better/simpler way??? >>> >>> (Btw, I would be very grateful for feedback on the form of my question >>> if you have any. Are there better/other forums? Could I present the >>> question or code in a another manner to help you understand better?) >>> >>> I am happy for any replies - on the list or backchannel. And I hope that >>> you feel that I have not misused your inbox (I may repeat this behavior...). >>> >>> Thanks, >>> Brjánn >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Clojure" group. >>> To post to this group, send email to clojure@googlegroups.com >>> Note that posts from new members are moderated - please be patient with >>> your first post. >>> To unsubscribe from this group, send email to >>> clojure+unsubscr...@googlegroups.com >>> For more options, visit this group at >>> http://groups.google.com/group/clojure?hl=en >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Clojure" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to clojure+unsubscr...@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> -- >> James Reeves >> booleanknot.com >> >> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clojure@googlegroups.com >> Note that posts from new members are moderated - please be patient with >> your first post. >> To unsubscribe from this group, send email to >> clojure+unsubscr...@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> --- >> You received this message because you are subscribed to the Google Groups >> "Clojure" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to clojure+unsubscr...@googlegroups.com. >> For more options, visit https://groups.google.com/d/optout. >> > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- James Reeves booleanknot.com -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
