Just want to mention DSE which is especially made to solve problems with mass insert and/updates in django:
http://pypi.python.org/pypi/dse/0.3.0 Regards, Thomas On Wed, Dec 29, 2010 at 1:22 PM, Derek <gamesb...@gmail.com> wrote: > Thanks Craig - that explains it very clearly indeed ("textbook answer"!) > > On 27 December 2010 16:50, W. Craig Trader <craig.tra...@gmail.com> wrote: >> >> Derek ... >> >> Quoting from http://docs.djangoproject.com/en/1.2/topics/db/transactions/: >>> >>> Django’s default behavior is to run with an open transaction which it >>> commits automatically when any built-in, data-altering model function is >>> called. For example, if you call model.save() or model.delete(), the change >>> will be committed immediately. >>> >>> This is much like the auto-commit setting for most databases. As soon as >>> you perform an action that needs to write to the database, Django produces >>> the INSERT/UPDATE/DELETE statements and then does the COMMIT. There’s no >>> implicit ROLLBACK. >> >> Using the Django transaction API, you can manually control transactions. >> In this case, Silva must be doing something like this: >> >> from django.db import transaction >> >> @transaction.commit_manually >> def viewfunc(request): >> >> >> >> ... >> # You can commit/rollback however and whenever you want >> i = 0 >> for x in data: >> ... save something ... >> i += 1 >> if i % 1000: >> >> transaction.commit() >> >> # One last commit to persist the last batch of saves >> transaction.commit() >> >> >> >> >> I will also note that the transaction API can be (and should be) used when >> you're writing management commands. >> >> - Craig - >> >> On Mon, Dec 27, 2010 at 05:29, derek <gamesb...@gmail.com> wrote: >>> >>> Can you explain what you mean by "manual commitments" (preferably with >>> a code example)? >>> >>> Thanks >>> Derek >>> >>> On Dec 26, 4:38 am, Silva Paulo <psdasi...@yahoo.com> wrote: >>> > OK. I found a "solution". >>> > Using manual commitments after lots of insertions (ex. >1000) makes the >>> > massive insertion very fast (at least for my needs). >>> > >>> > Thanks to all who answered. >>> > >>> > --- On Fri, 12/24/10, Silva Paulo <psdasi...@yahoo.com> wrote: >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > > From: Silva Paulo <psdasi...@yahoo.com> >>> > > Subject: Massive insertion of records >>> > > To: "newbie" <django-users@googlegroups.com> >>> > > Date: Friday, December 24, 2010, 6:26 PM >>> > > I need to do a massive insertion of >>> > > records in two tables "connectd" by "foreignkey". Is there a >>> > > way, using the Django db API, to do it. e=Foo(...);e.save() >>> > > seems too slow. >>> > >>> > > Thanks >>> > >>> > > >>> > >>> > > -- >>> > > You received this message because you are subscribed to the >>> > > Google Groups "Django users" group. >>> > > To post to this group, send email to django-us...@googlegroups.com. >>> > > To unsubscribe from this group, send email to >>> > > django-users+unsubscr...@googlegroups.com. >>> > > For more options, visit this group >>> > > athttp://groups.google.com/group/django-users?hl=en. >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Django users" group. >>> To post to this group, send email to django-us...@googlegroups.com. >>> To unsubscribe from this group, send email to >>> django-users+unsubscr...@googlegroups.com. >>> For more options, visit this group at >>> http://groups.google.com/group/django-users?hl=en. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To post to this group, send email to django-us...@googlegroups.com. >> To unsubscribe from this group, send email to >> django-users+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/django-users?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To post to this group, send email to django-us...@googlegroups.com. > To unsubscribe from this group, send email to > django-users+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-users?hl=en. > -- Mvh/Best regards, Thomas Weholt http://www.weholt.org -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
