Rodrigo, We went with RabbitMQ over ZeroMQ mostly because we were more familiar with it. As I understand it, ZeroMQ is less of a message-queueing system and more "sockets on steroids." Which one is best will depend pretty strongly on your particular usecase.
In our case, our messages are pretty simple: they're just mongodb docIds plus some extra data. We use the queue the message goes into as the "meta-data" for the message. If a docId appears in the "word-count" queue, for example, the Clojure workers know to calculate the avg word count stats for the given document. For us, security comes in the form of username/password authentication for RabbitMQ, plus blocking the ports to everyone save the Ruby and Clojure servers that need to communicate with it. If we were firing off messages with more sensitive data (e.g., passwords), we'd look at encryption then. The performance of the pipeline so far has been great. Ruby can offload the heavy processing to Clojure, so the user can continue working on the site without interruption. Clojure gets to focus on just the processing side, so we can optimize and scale it separately. As for portability, we started with everything on Heroku, but we've been slowly migrating to aws as our needs scale up. Our configuration makes this really easy: we can move the rabbitmq piece over one week, then the clojure the next, without changing anything but a few lines in a config file. Ron On Monday, September 9, 2013 7:47:54 AM UTC-7, rdelcueto wrote: > > Hey Ron, > Thanks for your response. Digging deeper into my question... > > When I read about the Torquebox Immutant duet, I thought it was > particularly interesting solution, because it was fairly easy to deploy and > both processes would live inside a JVM environment. I was impressed by how > Clojure data structures mapped to Ruby structures and vice-versa, it seemed > to provide a very clean and idiomatic messaging platform. Plus it would > provide tools for caching, clustering, and what not. Still I wasn't very > keen on the JRuby subject, since It's known to have compatibility issues > with certain gems. > > Yesterday while researching on the subject I found about ZeroMQ. Do you > have any particular reason to use RabbitMQ over other messaging libraries? > Are there any caveats to your interop model? > How portable is deploying a site using a messaging solution such as > RabbitMQ? > > I also found out about Google's Protocol Buffers, they seemed like a > lightweight solution to pass language agnostic data structures through the > messaging infrastructure. > Do messages need to be encapsulated somehow, or is this actually > unnecessary? How it's done in your case? > > Regarding security and sensible information interop; Should messages be > encrypted? Should they be encrypted as a whole message or partially (only > sensible data)? > What are the performance implications of this pipeline? Is the overhead > and footprint of such setup (Ruby + Messaging Broker + ClojureJVM) big > enough, for it to be worth thinking on writing everything in Clojure (using > the Luminus framework)? > > On Monday, September 9, 2013 8:10:41 AM UTC-5, Ron Toland wrote: >> >> At Rewryte, we use Rails for the web frontend and Clojure for the data >> processing backend for exactly the reasons you described. >> >> We use RabbitMQ to communicate between the two. This maintains separation >> between the two apps (no JRuby required), and lets us scale them both >> independently, while taking advantage of each language/framework's >> strengths. >> > -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
