On Jan 30, 2008 8:55 PM, Mark Green <[EMAIL PROTECTED]> wrote: > What ressources are held and wasted exactly? > Maintaining a number of open TCP connection is much cheaper > than creating/discarding them at a high rate.
Every connection that one Django application holds on to is a connection that something else can't use. > I agree that django's CGI-style mode of operation might make > implementation tricky (separate thread?) but you can't seriously > suggest that creating/discarding n connections per second would > be cheaper that maintaining, say, n*10 long-lived connections? If you use an external pooling manager you're not creating/discarding *database* connections, which would really hurt; you're creating/discarding connections to the connection manager. > Predictability is the keyword here. From the perspective of my database > (or pgpool instance) I want to be sure that the configured maximum > number of inbound connections can never be exceeded because clients > (such as django) should never get into the uncomfortable situation > of having to deal with a "connection refused". It's *your* keyword. The question from Django's perspective is whether it is of such all-consuming importance that Django must be completely re-architected to accommodate your use case. The answer from Django's perspective is "no". > "Fail-fast" as by django just doesn't work so well on the frontend. > Users don't like error pages at all, they're a surefire way to damage > your reputation. Yes, slow load times are bad too, but still a much more > comfortable position to be in during temporary rush hours. A slow-loading page is an error, to a user. Numerous usability studies demonstrate this. > Hmm. Dynamic adaption of the pool config is an interesting subject > (java c3p0 can actually do it at runtime, within limits, i think) but > totally out of my scope here. I think a fixed pool config would suffice > to achieve my goal of "graceful behaviour under load". No, it wouldn't. And you'd see that if you'd actually read what I'm saying here. > Oh c'mon. A connection pool is not so complicated. Compared to not having it? Yes, yes it is. > Well, I'd think that the constant flux of connections > causes more overhead than even the sloppiest implementation > of a pool ever could. Creating a TCP connection is not free > and then there's the DB handshake on top. Again, see above about connecting to an external pooling manager. > Out of curiosity: Does django create a new connection for each query or > for each http-request? For each request/response cycle. > And about code complexity... Yes, tying the thing to the django > execution model might take a bit of thinking. But the pool itself > should be a 20-liner, we're talking python, right? ;-) 20 lines of code to implement the pool, and complete re-architecting of a 32,000-line codebase so you'll have a place to put it. A winning trade this is not. > Oh c'mon again... You're saying it *wouldn't* tie the application layer to knowing about the database layer? > Sounds like a win to me, what kind of flexibility would be lost here? Suppose I have my Django site running on one physical web server, and I configure a connection pool within Django and do all the math to make it fail gracefully. Now, suppose my traffic grows to where I need a second physical web server. Whoops, time to re-calculate the Django configuration. If I instead use a pooling connection manager external to Django, I just drop another web node into place and keep right on trucking, with zero reconfiguration and zero downtime. That, to my mind, qualifies as "flexibility". > Well, doing it *this* way is obviously a bad idea. If django were to > persist anything across requests (such as a connection pool) then > ofcourse a long-running thread/process seems like the logical choice. Yes, and once you're there why not keep the application layer simple by having that long-running process not be part of it? > Although I still wonder if the mod_python stuff really doesn't allow for > any kind of persistence or if django just chose not to use it? You *can* persist things quite easily on a per-process basis, and Django does persist some things, just not the database connection. > As said, the particular problem of "users getting error pages instead of > delays" can unfortunately not be solved externally. Yes, it can. We do it already in production. Here's how: 1. Set up your web nodes. 2. Set up Perlbal in front of them (even if you have only one web node, you'll want a load balancer for any sort of serious traffic, and Perlbal's a best of breed tool for this sort of stack). 3. When you hit a max connections and a web node errors out, Perlbal detects this and routes to another, effectively re-trying the request. And voila: graceful retry without cluttering up the application layer. And in the time you've spent arguing for Django to be fundamentally rearchitected to deal with this already-solved problem, you could have learned how to do that by following up on lots of publicly-available deployment stories which talk about this sort of thing. The Java way is not the only way, and it is not the only good way. Django follows a different design and has a different architecture, and -- again -- attempting to impose assumptions from the Java world will usually end in frustration because -- for no other reason than that you'd already decided in advance how you wanted the solution to look -- you'll end up missing simple and useful solutions. -- "Bureaucrat Conrad, you are technically correct -- the best kind of correct." --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~----------~----~----~----~------~----~------~--~---
