Re: [elephant-devel] Next issue, this time with Postmodern backend

2008-02-28 Thread Robert L. Read
On Thu, 2008-02-28 at 16:32 +0100, Leslie P. Polzer wrote: > > I'm imagining something that spins up 100 simultaneous threads and does > > something to elephant from within each thread. Following the mantra > > "red, green, refactor", it would be lovely to see things seize up and > > then see

Re: [elephant-devel] Next issue, this time with Postmodern backend

2008-02-28 Thread Robert L. Read
On Thu, 2008-02-28 at 09:45 +0100, Leslie P. Polzer wrote: > Dear Robert, > > you must really be a fan of TDD :) Yes. I have had the pleasure of pair programming with Kent Beck himself. But more to the point, the test suite in Elephant is one of our strongest assets. Where it is weak (like in

Re: [elephant-devel] Next issue, this time with Postmodern backend

2008-02-28 Thread Ian Eslick
Sounds like we want to add bordeax threads to the elephant dependency list so we have a proper interface to threads for testing and some of the other stuff (thread-alive-p) that we currently hack up in the elephant utils. Any objections? Ian On Feb 28, 2008, at 10:32 AM, Leslie P. Polzer

Re: [elephant-devel] Next issue, this time with Postmodern backend

2008-02-28 Thread Leslie P. Polzer
> I'm imagining something that spins up 100 simultaneous threads and does > something to elephant from within each thread. Following the mantra > "red, green, refactor", it would be lovely to see things seize up and > then see that your patch fixes it. It would also be a starting point > f

Re: [elephant-devel] Next issue, this time with Postmodern backend

2008-02-28 Thread Leslie P. Polzer
Dear Robert, you must really be a fan of TDD :) I'll see what I can do. Fortunately, the thread tests should be trivial to implement, as opposed to multi-process tests (but we have this in GRAND-PRIX). To make the tests easily accessible as a confidence-inspiring device for the everyday develop

Re: [elephant-devel] Next issue, this time with Postmodern backend

2008-02-27 Thread Robert L. Read
Dear Leslie, Thank you for this patch. I will try to test it by this weekend, unless Ian wants to. However, if you will forgive me being dogmatic, it would be even better if this had a test case with it. For example, I don't think we have any good examples of producing lots of th

Re: [elephant-devel] Next issue, this time with Postmodern backend

2008-02-27 Thread Leslie P. Polzer
> You know, I think people haven't run into this problem because may web > servers maintain and reuse pools of worker threads - thus the # of > connections = max concurrent threads rather than # of sessions/requests. Hunchentoot seems to spawn a new worker on every incoming request, or at least g

Re: [elephant-devel] Next issue, this time with Postmodern backend

2008-02-27 Thread Ian Eslick
You know, I think people haven't run into this problem because may web servers maintain and reuse pools of worker threads - thus the # of connections = max concurrent threads rather than # of sessions/requests. Definitely good to have this in there: dangling sockets/file refs are bad! Ian

Re: [elephant-devel] Next issue, this time with Postmodern backend

2008-02-27 Thread Leslie P. Polzer
> Elephant-utils has some of the basic thread stuff we need - only need > to import one or two operators. If we add much more we'll add > bordeaux as a dependency. Do you want to wrap this up in a bow so we > can commit it? (Did you test by adding this to the connection > creation routine in pm-

Re: [elephant-devel] Next issue, this time with Postmodern backend

2008-02-27 Thread Ian Eslick
Elephant-utils has some of the basic thread stuff we need - only need to import one or two operators. If we add much more we'll add bordeaux as a dependency. Do you want to wrap this up in a bow so we can commit it? (Did you test by adding this to the connection creation routine in pm-co

Re: [elephant-devel] Next issue, this time with Postmodern backend

2008-02-27 Thread Leslie P. Polzer
> Ah, you have alot of short-lived threads? A quick look at pm- > controller.lisp seems to indicate that pm-postmodern does not close > connections when the thread closes. Err... amazing that no one else experienced issues with this. After all not only people using short-lived threads will have

Re: [elephant-devel] Next issue, this time with Postmodern backend

2008-02-26 Thread Ian Eslick
Ah, you have alot of short-lived threads? A quick look at pm- controller.lisp seems to indicate that pm-postmodern does not close connections when the thread closes. Perhaps you can add a function to do this explicitly or better, simply scan through active threads and cull any connections f

Re: [elephant-devel] Next issue, this time with Postmodern backend

2008-02-26 Thread Leslie P. Polzer
> Sounds like you're opening to many connections to the db. How many > threads do you have? Each thread opens it's own socket to the server > so if you have 100 threads, you have 100 connections to the server. Well, like I pointed out in my first mail, the threads are very few. They make peak t

Re: [elephant-devel] Next issue, this time with Postmodern backend

2008-02-26 Thread Ian Eslick
Sounds like you're opening to many connections to the db. How many threads do you have? Each thread opens it's own socket to the server so if you have 100 threads, you have 100 connections to the server. This is probably because the client library is not thread-safe. But I'm speculating

Re: [elephant-devel] Next issue, this time with Postmodern backend

2008-02-26 Thread Leslie P. Polzer
> There must be something fishy either with PostgreSQL 8 (tested 8.2.6, 8.1.11) > or Postmodern. I tend to put it on the latter, because GRAND-PRIX with CLSQL > doesn't show any problems, nor do any workers accumulate. Update: with CLSQL and my application, it's the same thing. But I can't see wh

[elephant-devel] Next issue, this time with Postmodern backend

2008-02-26 Thread Leslie P. Polzer
Internal Server Error Database error 53300: connection limit exceeded for non-superusers 0: (BACKTRACE 536870911 #) 1: (HUNCHENTOOT:GET-BACKTRACE #) 2: ((LAMBDA (COND)) #) 3: ((LAMBDA (COND)) #) 4: (SIGNAL #) 5: (ERROR CL-POSTGRES:DATABASE-ERROR) 6: (CL-POSTGRES::GET-ERROR #) 7: (CL-POSTGRES::GET