Re: Django scalability question

2006-03-01 Thread hsitz
Okay, reading the new transaction docs a little closer it looks like you've implemented method (2) in my message above. Seems to me like this is the necessary method, the only sane thing to do in web context. However, since the transactions apply only in short-lived context of when user is savi

Re: Django scalability question

2006-03-01 Thread hsitz
Jacob -- I'm a mostly non-web programmer (with experience in non-web n-tier apps) trying to figure out exactly what sort of transaction support this is. Can you clarify for me? Here are the two types of transactions I'm thinking people might want: (1) Transaction is begun when a client makes a

Re: Django scalability question

2006-03-01 Thread Jacob Kaplan-Moss
... And now it's done -- see http://code.djangoproject.com/browser/django/branches/magic-removal/docs/transactions.txt for the docs and http://code.djangoproject.com/changeset/2457 for the code. This should help a lot for high-volume writes, I suspect. Jacob --~--~-~--~~--

Re: Django scalability question

2006-03-01 Thread hugo
>That's good to know, but as you have stated previously, transactions >alone won't solve this problem. There still needs to be some concept of >versioning on top of the transactions, which isn't trivial >(arguably/perhaps even harder to do right than supporting transaction >wrappers). As Adrian w

Re: Django scalability question

2006-03-01 Thread ZebZiggle
I agree Herb, you need transactions as the basis for making this work. But, as you have stated you need other infrastructure to make the optimistic locking work. Either of the two approaches you mentioned make sense (although there are some issues in using timestamps). Again, Hibernate is a good m

Re: Django scalability question

2006-03-01 Thread Jacob Kaplan-Moss
Actually I'm about the check this in :) I've got a few things to fix before I do, but expect Django to have transaction support by noon. Jacob --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" gr

Re: Django scalability question

2006-03-01 Thread James Bennett
On 3/1/06, Ned Batchelder <[EMAIL PROTECTED]> wrote: > It is too bad that Django doesn't support transactions. If you need > them, then you need them badly, and you should find a framework that > supports them to build your application. Many web applications don't > require them, and Django is g

Re: Django scalability question

2006-03-01 Thread Ned Batchelder
If these things are problems for your application, you need transactions, period. Anything else you cobble together without transactions will merely be a buggy implementation of half of a real transaction system. You'll keep encountering problems until you finally embrace existing transactio

Re: Django scalability question

2006-03-01 Thread hsitz
"Think distributed: two requests updating the same data concurrently. Last write wins. Data might not be what you expect, as you can't make sure that you have the version you directly read before updating. This is the simple scenario. Often this doesn't matter, as "last write wins" is quite accept

Re: Django scalability question

2006-03-01 Thread hsitz
ZebZiggle -- Sorry, if you're not talking about any sort of auto-refresh at all, and are just talking about fact that in Django the second user's object will automatically overwrite the object that the first user just saved in the db, without any warning or error message, then I agree this is a p

Re: Django scalability question

2006-03-01 Thread hsitz
"That's good to know, but as you have stated previously, transactions alone won't solve this problem. There still needs to be some concept of versioning on top of the transactions, which isn't trivial (arguably/perhaps even harder to do right than supporting transaction wrappers). Perhaps we need

Re: Django scalability question

2006-03-01 Thread Adrian Holovaty
On 3/1/06, Arthur <[EMAIL PROTECTED]> wrote: > The other day I visited Rob Curley in Naples and got a good look at the > tremendous work Rob (and the whole ljworld team) have done with school > sports databases. Individual stats on a thousand players. > > My question is, what happens if I try to u

Re: Django scalability question

2006-03-01 Thread Arthur
The other day I visited Rob Curley in Naples and got a good look at the tremendous work Rob (and the whole ljworld team) have done with school sports databases. Individual stats on a thousand players. My question is, what happens if I try to use Django to do the same thing at the New York Daily N

Re: Django scalability question

2006-02-28 Thread ZebZiggle
That's good to know, but as you have stated previously, transactions alone won't solve this problem. There still needs to be some concept of versioning on top of the transactions, which isn't trivial (arguably/perhaps even harder to do right than supporting transaction wrappers). Perhaps we need

Re: Django scalability question

2006-02-28 Thread hugo
>That said, it is nice to see someone admit that this is a problem. :-) Actually _that_ was never the problem: the ticket on transactions (whose absence is the actual culprit in this problem area) is ticket #9 :-) bye, Georg --~--~-~--~~~---~--~~ You received th

Re: Django scalability question

2006-02-28 Thread ZebZiggle
Hmm, I don't think you could get a cycle short enough to avoid the race conditions described above. I agree that keeping objects in memory will not help the problem (as per my original post), but constantly freshening the object won't work either. Perhaps single-user is a an unqualified overstate

Re: Django scalability question

2006-02-28 Thread Ivan Sagalaev
ZebZiggle wrote: >I'm sure there are many very large websites using Django, but from what >I see many are newspaper-style (many reads, few if any writes except by >the admins). I'd be curious how may sites are doing dynamic updates by >many concurrent users? > It's still not a problem for a typic

Re: Django scalability question

2006-02-28 Thread ZebZiggle
Hugo/Georg is exactly correct. I'm sure there are many very large websites using Django, but from what I see many are newspaper-style (many reads, few if any writes except by the admins). I'd be curious how may sites are doing dynamic updates by many concurrent users? In a read-only / content / p

Re: Django scalability question

2006-02-28 Thread hugo
>Of *course* you'd expect that obj2.data != obj1.data -- Django's not >going to be able to hide the fact that you're using a database from you >(nor should it). Think distributed: two requests updating the same data concurrently. Last write wins. Data might not be what you expect, as you can't ma

Re: Django scalability question

2006-02-28 Thread James Bennett
On 2/28/06, ZebZiggle <[EMAIL PROTECTED]> wrote: > Sorry, but I don't think Django will work in anything but a single user > environment, or multi-users only doing read-only access. Unless someone > from the Django team can clarify. Please read my thread on this issue > ... it's a big concern for

Re: Django scalability question

2006-02-28 Thread Jacob Kaplan-Moss
On Feb 28, 2006, at 7:49 AM, ZebZiggle wrote: > Sorry, but I don't think Django will work in anything but a single user > environment, or multi-users only doing read-only access. Unless someone > from the Django team can clarify. I'd be glad to: I have three web servers hitting the same database

Re: Django scalability question

2006-02-28 Thread ZebZiggle
Sorry, but I don't think Django will work in anything but a single user environment, or multi-users only doing read-only access. Unless someone from the Django team can clarify. Please read my thread on this issue ... it's a big concern for me. http://groups.google.com/group/django-users/browse_f

Re: Django scalability question

2006-02-27 Thread David Pratt
Hi Adrian. Many thanks for your reply. This helps to answer my question and it is pretty much as I expected. Do you have any additional infomation on the sort of volume ChicagoCrime and others are handling and how they are generally coping. Regards, David Adrian Holovaty wrote: > On 2/27/06,

Re: Django scalability question

2006-02-27 Thread Adrian Holovaty
On 2/27/06, David Pratt <[EMAIL PROTECTED]> wrote: > What does scalability look like for Django once you get to the limit of > your initial serving capacity. I would be interested in what is occuring > with some of these large newspaper sites to handle the load. Hi David, The general way of scal

Django scalability question

2006-02-27 Thread David Pratt
I am weighing Django and Zope3 for a project for a higher volume site that has to scale for future. It would seem that with Django app could be potentially faster wih mod_python than Zope3 but Zope has a clear path for scalability with ZEO (so multiple application servers ahead of one or more