Hi. So I wanted to see how easy it would be to implement core.async on top of Pulsar. It turned out to be quite trivial, but the exercise shows the different approaches of the two projects.
First, Pulsar <https://github.com/puniverse/pulsar> is the Clojure API to Quasar <https://github.com/puniverse/quasar>, which, in turn, is a new concurrency framework for the JVM (introduced here<http://blog.paralleluniverse.co/post/49445260575/quasar-pulsar> ). At its heart it is an implementation of true lightweight threads (fibers) on the JVM. Fibers are created by instrumenting the bytecode of selected methods, turning them into continuations and running them in a ForkJoinPool. Then, Quasar/Pulsar provide communication mechanism in the form of Go-like channels and dataflow variables. The last layer is Erlang-like actors, complete with selective receive plus other OTP goodies like gen-server, supervisors and more. We are currently working on version 0.2, which we will hopefully release at the end of the week, whose main feature is clustering (channels and actors can communicate over the network). Anyway, this is the core.async implementation on top of Pulsar<https://github.com/puniverse/pulsar/blob/master/src/main/clojure/co/paralleluniverse/pulsar/async.clj>. I haven't implemented alts yet, because that's the only feature that would take more than a couple of lines; I might have the time to do it tomorrow. core.async turns Clojure expressions into continuations at the Clojure level, while Pulsar works at the bytecode level, so this alone has a few consequences. First, the Pulsar implementation can't run ClojureScript. Second, several limitations put in place by core.async become unnecessary, first and foremost is the distinction between <!! and <!, and >! and >!!. In the Pulsar implementation you can use either the bang or the double-bang form inside or outside a go block. It makes no difference. Then, we have the channels. Pulsar's channels are lock-free, and are single-consumer by default. It seems to me that channels should be single-consumer, but I'd like to hear other opinions on the matter. Making channels single-consumer not only saves a couple of CASs, it also opens the door to some operations that would require locks without this assumption (though, just to be sure, the channels in our core.async implementation are all multi-producer multi-consumer; I'm just not sure that's the right choice). Multiple consumers are also a problem when it comes to distributing channels over the network (for now, only our single-consumer channels can be distributed). Distribution brings us to alts. While easy to implement with Quasar/Pulsar on a single machine, I'm not sure how easily they can be distributed. If you have an alt possibly sending a message to one of two distributed queues, it will require quite a few roundtrips, and possibly even a consensus protocol (though maybe not) to make sure the message is sent to one, and only one channel. Not any better is the problem on waiting on several distributed channels - that requires a distributed transaction. One of the things I like most about Quasar/Pulsar is how synchronization is carried out at the fiber level. Because fibers run in a FJPool, true thread-blocking locks are a no-no (they can wreak havoc on FJ). But Quasar abstracts threads and fibers into something called "strands"; a strand is simply a fiber or a thread. This gives a single, identical API, for dealing with both fibers and threads. As a result, converting thread-synchronization constructs to such that can work with both fibers and threads is almost automatic. When a thread parks, it actually parks, but when a fiber parks it is simply descheduled off the FJPool, and when it "unparks" it is re-forked on the pool and continues right where it left off. We have big plans for Quasar/Pulsar, but I don't want to spoil all the surprises. I'd love to hear some feedback. Ron -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
