Re: ANN: Aleph, an asynchronous web server

2010-07-09 Thread Anders Rune Jensen
On Wed, Jul 7, 2010 at 11:15 AM, Zach Tellman wrote: > At the Bay Area user group meeting in June, there was a very > interesting discussion about how to best use Clojure's concurrency > primitives to field large numbers of concurrent requests, especially > in a long-poll/push type application.  W

Re: ANN: Aleph, an asynchronous web server

2010-07-09 Thread Peter Schuller
> Maybe what I said makes less sense in the case of NIO vs blocking with > threads - I've mainly been working with Intel Threading Building Blocks > lately, where the cost of cache cooling is very real. For that reason (and > the others mentioned - context switches and lock preemption), Intel > Thr

Re: ANN: Aleph, an asynchronous web server

2010-07-09 Thread David Nolen
On Fri, Jul 9, 2010 at 9:44 AM, Anders Rune Jensen < anders.rune.jen...@gmail.com> wrote: > On Fri, Jul 9, 2010 at 3:09 PM, David Nolen > wrote: > > On Fri, Jul 9, 2010 at 8:44 AM, Anders Rune Jensen > > wrote: > >> > >> Very interesting! > >> > >> I've been following the thread with great inter

Re: ANN: Aleph, an asynchronous web server

2010-07-09 Thread James Reeves
On 9 July 2010 14:09, David Nolen wrote: > I'm curious how you ran that test. With ab running 10 clients for 1 second I > see ~4000-5000 req/s using Compojure 0.4.0. With aleph I see ~8000-9000 > req/s. I also had a quick chat with Zach Tellman and it sounds like he > hasn't done much in the way o

Re: ANN: Aleph, an asynchronous web server

2010-07-09 Thread Anders Rune Jensen
On Fri, Jul 9, 2010 at 3:09 PM, David Nolen wrote: > On Fri, Jul 9, 2010 at 8:44 AM, Anders Rune Jensen > wrote: >> >> Very interesting! >> >> I've been following the thread with great interest and did a quick >> performance test today comparing standard compojure with jetty against >> aleph and

Re: ANN: Aleph, an asynchronous web server

2010-07-09 Thread Daniel Kersten
Maybe what I said makes less sense in the case of NIO vs blocking with threads - I've mainly been working with Intel Threading Building Blocks lately, where the cost of cache cooling is very real. For that reason (and the others mentioned - context switches and lock preemption), Intel Threading Bui

Re: ANN: Aleph, an asynchronous web server

2010-07-09 Thread gary b
On Jul 8, 2:26 pm, Antoni Batchelli wrote: > Also, in some instances with NIO you can even work directly > with kernel buffers, and so the network data doesn't even need > to be copied from the kernel space into the user space. I assume that you are referring to NIO direct byte buffers. A thread

Re: ANN: Aleph, an asynchronous web server

2010-07-09 Thread David Nolen
On Fri, Jul 9, 2010 at 8:44 AM, Anders Rune Jensen < anders.rune.jen...@gmail.com> wrote: > Very interesting! > > I've been following the thread with great interest and did a quick > performance test today comparing standard compojure with jetty against > aleph and netty. I get around 4500 req/s w

Re: ANN: Aleph, an asynchronous web server

2010-07-09 Thread Anders Rune Jensen
On Wed, Jul 7, 2010 at 11:15 AM, Zach Tellman wrote: > At the Bay Area user group meeting in June, there was a very > interesting discussion about how to best use Clojure's concurrency > primitives to field large numbers of concurrent requests, especially > in a long-poll/push type application.  W

Re: ANN: Aleph, an asynchronous web server

2010-07-08 Thread Greg
Whoops. s/Raoul/Peter Schuller/. On Jul 8, 2010, at 7:03 PM, Greg wrote: > I hope it didn't sound like I was saying threads are *always* bad, as I > definitely don't think that. :-p > > Your link to the epoll + threads document is probably the best way to go > (that I'm aware of), to address

Re: ANN: Aleph, an asynchronous web server

2010-07-08 Thread Greg
I hope it didn't sound like I was saying threads are *always* bad, as I definitely don't think that. :-p Your link to the epoll + threads document is probably the best way to go (that I'm aware of), to address any of the issues that Raoul brought up w.r.t. long operations in between the events

Re: ANN: Aleph, an asynchronous web server

2010-07-08 Thread Greg
Interesting link! Unfortunately the link to the PDF was broken, here's one that works: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.101.7987&rep=rep1&type=pdf On Jul 8, 2010, at 6:48 PM, Raoul Duke wrote: > can't we all just get along? > > http://lambda-the-ultimate.org/node/1435 >

Re: ANN: Aleph, an asynchronous web server

2010-07-08 Thread Raoul Duke
On Thu, Jul 8, 2010 at 2:36 PM, Greg wrote: > A fundamental understanding of the difference between threads and > kqueue/epoll (which power NIO) should clear up anyone's misgivings about > evented servers. They are clearly more scalable, it is no contest. oh Erlang, were art thou? -- You rece

Re: ANN: Aleph, an asynchronous web server

2010-07-08 Thread Greg
Great response Antoni. A fundamental understanding of the difference between threads and kqueue/epoll (which power NIO) should clear up anyone's misgivings about evented servers. They are clearly more scalable, it is no contest. - Greg On Jul 8, 2010, at 5:26 PM, Antoni Batchelli wrote: > On

Re: ANN: Aleph, an asynchronous web server

2010-07-08 Thread Raoul Duke
can't we all just get along? http://lambda-the-ultimate.org/node/1435 -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient wit

Re: ANN: Aleph, an asynchronous web server

2010-07-08 Thread Peter Schuller
> Under heavy load, this can be quite costly, especially if each request > requires non-trivial processing (ie, enough to make time-slicing kick > in). This doesn't really jive with reality as far as I can tell; if anything it is the exact opposite of reality. If you're doing significant work in b

Re: ANN: Aleph, an asynchronous web server

2010-07-08 Thread Antoni Batchelli
On Jul 7, 2010, at 8:47 PM, gary b wrote: > This blog post presents data showing that threading is faster than > NIO: > http://mailinator.blogspot.com/2008/02/kill-myth-please-nio-is-not-faster-than.html > I would not consider this article to be the definitive answer to the question of NIO vs th

Re: ANN: Aleph, an asynchronous web server

2010-07-08 Thread Wilson MacGyver
thank you! On Thu, Jul 8, 2010 at 3:09 PM, David Nolen wrote: > On Thu, Jul 8, 2010 at 3:02 PM, Wilson MacGyver wrote: >> >> for the hello world test, you are using the helloworld from >> front page of node.js at http://nodejs.org/ >> right? >> >> how did you setup the clojure one? >> >> was it

Re: ANN: Aleph, an asynchronous web server

2010-07-08 Thread David Nolen
On Thu, Jul 8, 2010 at 12:55 PM, Miki wrote: > Hello David, > > > while aleph (via Netty) is easily getting 600-700 rq/s > > even if I'm writing to a database (CouchDB). > Can you share the code for this? > > All the best, > -- > Miki > I don't have a standalone example at the moment. I might tr

Re: ANN: Aleph, an asynchronous web server

2010-07-08 Thread David Nolen
On Thu, Jul 8, 2010 at 3:02 PM, Wilson MacGyver wrote: > for the hello world test, you are using the helloworld from > front page of node.js at http://nodejs.org/ > right? > > how did you setup the clojure one? > > was it what you posted before? > I wrote a blog post about it here with the code

Re: ANN: Aleph, an asynchronous web server

2010-07-08 Thread Dan Kersten
There are more rasons to want to avoid using threads than memory. Besides the obvious cost of creating and destroying threads (which is reduced or removed by using thread pools), you also have the cost of time slicing once you have more software threads than hardware threads: there is the obvious c

Re: ANN: Aleph, an asynchronous web server

2010-07-08 Thread Wilson MacGyver
for the hello world test, you are using the helloworld from front page of node.js at http://nodejs.org/ right? how did you setup the clojure one? was it what you posted before? (defn hello-world [request] (future (Thread/sleep 1) (respond! request {:status 200

Re: ANN: Aleph, an asynchronous web server

2010-07-08 Thread Miki
Hello David, > while aleph (via Netty) is easily getting 600-700 rq/s > even if I'm writing to a database (CouchDB). Can you share the code for this? All the best, -- Miki -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send e

Re: ANN: Aleph, an asynchronous web server

2010-07-08 Thread gary b
re: memory use If the number of concurrent requests is small, then the memory used by thread per request is usually not an issue. When implementing long polling, the number of concurrent requests can be very large. Sharing threads between requests in a long polling server can result in significa

Re: ANN: Aleph, an asynchronous web server

2010-07-07 Thread Greg
"well I think the main advantage" is memory. :-) Theoretically (I think), thread-per-connection servers can be very close to matching asynchronous servers in throughput, but they definitely require much more RAM to do so. RAM is one of the more expensive commodities to come by on VPS and cloud

Re: ANN: Aleph, an asynchronous web server

2010-07-07 Thread David Nolen
On Wed, Jul 7, 2010 at 2:10 PM, James Reeves wrote: > The main advantage of a non-blocking server is that you're don't use > up a thread waiting for an event (such as the user sending data, or > some other external trigger). > > - James I think the main advantage of a non-blocking server is thro

Re: ANN: Aleph, an asynchronous web server

2010-07-07 Thread Pedro Henriques dos Santos Teixeira
On Wed, Jul 7, 2010 at 3:10 PM, James Reeves wrote: > On 7 July 2010 19:04, David Nolen wrote: >> So something like this: >> (defn hello-world [request] >>   (future >>    (Thread/sleep 1) >>    (respond! request >>              {:status 200 >>               :headers {"Content-Type" "text/html"}

Re: ANN: Aleph, an asynchronous web server

2010-07-07 Thread Zach Tellman
On Jul 7, 11:17 am, David Nolen wrote: > On Wed, Jul 7, 2010 at 2:10 PM, James Reeves wrote: > > > > > > > On 7 July 2010 19:04, David Nolen wrote: > > > So something like this: > > > (defn hello-world [request] > > >   (future > > >    (Thread/sleep 1) > > >    (respond! request > > >        

Re: ANN: Aleph, an asynchronous web server

2010-07-07 Thread David Nolen
On Wed, Jul 7, 2010 at 2:10 PM, James Reeves wrote: > On 7 July 2010 19:04, David Nolen wrote: > > So something like this: > > (defn hello-world [request] > > (future > >(Thread/sleep 1) > >(respond! request > > {:status 200 > > :headers {"Content-Type" "text/

Re: ANN: Aleph, an asynchronous web server

2010-07-07 Thread James Reeves
On 7 July 2010 19:04, David Nolen wrote: > So something like this: > (defn hello-world [request] >   (future >    (Thread/sleep 1) >    (respond! request >              {:status 200 >               :headers {"Content-Type" "text/html"} >               :body "Hello world!"}))) > Is non-blocking and

Re: ANN: Aleph, an asynchronous web server

2010-07-07 Thread David Nolen
On Wed, Jul 7, 2010 at 1:43 PM, Zach Tellman wrote: > Developers are still required to "participate" in the NIO design, in > that blocking calls in the request handler need to be avoided to reap > the full benefits. Netty provides a lot of nice abstractions over > NIO, but kind of punts on how t

Re: ANN: Aleph, an asynchronous web server

2010-07-07 Thread Zach Tellman
On Jul 7, 8:28 am, ngocdaothanh wrote: > > For some reason I couldn't get 3.2.1.Final to come in via maven. > > I think you need to add this to project.clj: > :repositories [["jboss" "http://repository.jboss.org/nexus/content/ > groups/public/"]] > Thanks, I'll give that a try. > > What qualitie

Re: ANN: Aleph, an asynchronous web server

2010-07-07 Thread Zach Tellman
On Jul 7, 9:12 am, David Nolen wrote: > On Wed, Jul 7, 2010 at 5:15 AM, Zach Tellman wrote: > > With this in mind, I decided to make the thinnest possible wrapper > > around Netty such that a person could play around with alternate ways > > to use Clojure effectively.  The result can be found at

Re: ANN: Aleph, an asynchronous web server

2010-07-07 Thread ngocdaothanh
> For some reason I couldn't get 3.2.1.Final to come in via maven. I think you need to add this to project.clj: :repositories [["jboss" "http://repository.jboss.org/nexus/content/ groups/public/"]] > What qualities is it missing that make the bang misleading? I thinks ! means something dangerous

Re: ANN: Aleph, an asynchronous web server

2010-07-07 Thread David Nolen
On Wed, Jul 7, 2010 at 5:15 AM, Zach Tellman wrote: > With this in mind, I decided to make the thinnest possible wrapper > around Netty such that a person could play around with alternate ways > to use Clojure effectively. The result can be found at > http://github.com/ztellman/aleph. I played

Re: ANN: Aleph, an asynchronous web server

2010-07-07 Thread Zach Tellman
On Jul 7, 2:57 am, ngocdaothanh wrote: > > [org.jboss.netty/netty "3.2.0.BETA1"] > > Netty 3.2.1.Final has been released. > > I think the ! mark in "respond!" is kind of misleading. Why not change > it to "arespond"? For some reason I couldn't get 3.2.1.Final to come in via maven. I didn't want

Re: ANN: Aleph, an asynchronous web server

2010-07-07 Thread ngocdaothanh
> [org.jboss.netty/netty "3.2.0.BETA1"] Netty 3.2.1.Final has been released. I think the ! mark in "respond!" is kind of misleading. Why not change it to "arespond"? -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email t

Re: ANN: Aleph, an asynchronous web server

2010-07-07 Thread David Nolen
On Wed, Jul 7, 2010 at 5:15 AM, Zach Tellman wrote: > At the Bay Area user group meeting in June, there was a very > interesting discussion about how to best use Clojure's concurrency > primitives to field large numbers of concurrent requests, especially > in a long-poll/push type application. W

ANN: Aleph, an asynchronous web server

2010-07-07 Thread Zach Tellman
At the Bay Area user group meeting in June, there was a very interesting discussion about how to best use Clojure's concurrency primitives to field large numbers of concurrent requests, especially in a long-poll/push type application. We didn't arrive at any solid conclusion, but it was clear to e