Well, you can inspect the object and see it's primary key. Surely that means the INSERT is completed? So the workflow goes like this:
foo = Foo() foo.save() foo.pk # new primary key is available submit_gearman_task(foo.pk) Then in the Gearman worker: foo = Foo.objects.get(pk=foo_pk) # this causes a Foo.DoesNotExist exception That's what I don't understand. The primary key is assigned and available in the application layer in one place but then the lookup using that primary key fails in another place. Maybe it has something to do with the fact that the database is set to REPEATABLE-READ isolation level versus READ-COMMITTED. Still investigating... On Tuesday, May 28, 2013 1:57:03 PM UTC-4, Christophe Pettus wrote: > > I'll admit my MySQL skills are a bit rusty, but can you examine the > database logs to confirm that the session is in fact doing the INSERT and a > commit before the SELECT? > > On May 28, 2013, at 10:52, Chris Conover <clc...@gmail.com <javascript:>> > wrote: > > It's not really feasible to move this project to PostgreSQL. > > I honestly just don't understand the problem. According to the Django > documentation, the ORM functions in autocommit mode by default. In other > words, the changes should be committed as soon as the query is complete, > right? So after an INSERT, you should be able to do a SELECT and the data > should be available. In the code written, the SELECT is definitely coming > after the INSERT. The code is not complex. I'm setting up a local test to > simulate the load on the production system to see if I can reproduce the > issue. > > On Tuesday, May 28, 2013 1:33:10 PM UTC-4, Christophe Pettus wrote: >> >> In the meantime, you can use this: >> >> https://github.com/Xof/xact >> >> On May 28, 2013, at 10:29, Chris Conover <clc...@gmail.com> wrote: >> >> That seems to only be available in the dev version of Django (now 1.6 >> alpha). >> >> On Wednesday, May 22, 2013 4:43:29 AM UTC-4, Christian Schmitt wrote: >>> >>> if you want transactions you need to do : >>> >>> with transaction.atomic(): >>> >>> >>> like described here: >>> https://docs.djangoproject.com/en/dev/topics/db/transactions/ >>> >>> Am Mittwoch, 22. Mai 2013 10:40:15 UTC+2 schrieb Christian Schmitt: >>>> >>>> In Django the normal behavior should be that when you do a save() it >>>> will automatically commit() your query's to the database. >>>> so that in obj.save() you should just could access the pk with obj.idafter >>>> you did a obj.save(). >>>> If you want to maybe stop the commit you need to do a obj = >>>> obj.save(commit=False), then you could add some things to your obj and >>>> commit/save it later. >>>> But as i understood you already do a obj.save() but it doesn't commit >>>> correctly? Maybe you should just try a PostgreSQL database for testing, >>>> since I'm not having trouble looking up objects after i saved it. >>>> >>>> I often do things like: >>>> obj.save() >>>> return HttpResponseRedirect(reverse('view', {'pk': obj.id})) >>>> and i never run into any exception >>>> >>>> >>>> Am Dienstag, 21. Mai 2013 23:20:53 UTC+2 schrieb Chris Conover: >>>>> >>>>> Calling transaction.commit() after object.save results in >>>>> a TransactionManagementError. I mentioned at the end that I am using >>>>> MySQL >>>>> (5.5.27). The issue is not that the Gearman workers are having trouble >>>>> saving their transactions, it's that they are having trouble looking up >>>>> the >>>>> incoming object. I'm assuming the view and workers are separate >>>>> transactions since I don't see how they could be connected -- though I'm >>>>> looking into this. >>>>> >>>>> On Tuesday, May 21, 2013 1:05:54 PM UTC-4, Tom Evans wrote: >>>>>> >>>>>> On Tue, May 21, 2013 at 4:23 PM, Chris Conover <clc...@gmail.com> >>>>>> wrote: >>>>>> > Hello, >>>>>> > >>>>>> > I'm having an issue looking up objects immediately after they are >>>>>> saved and >>>>>> > am wondering if anyone has any advice on how to address the >>>>>> problem. >>>>>> > Specifically, I'm saving an object in a view (nothing fancy, a >>>>>> simple >>>>>> > save()) and then kicking off a Gearman task to do some operations >>>>>> on that >>>>>> > saved object in the background. I pass the newly created object's >>>>>> PK as data >>>>>> > to the Gearman worker which then does a simple >>>>>> Objects.objects.get(pk=PK). >>>>>> > However, almost all of the time this lookup operation fails with an >>>>>> > DoesNotExist exception. I believe the problem has to do with >>>>>> transactions. >>>>>> > Namely, the read in the Gearman worker is happening before the >>>>>> write from >>>>>> > the view is committed to the database. I've tried several things >>>>>> including >>>>>> > refactoring the saving code to be wrapped in a >>>>>> > @transaction.commit_on_success block, moving the worker submission >>>>>> to >>>>>> > post_save and adding a long delay before the Gearman worker does >>>>>> the lookup. >>>>>> > Nothing really seems to solve the problem completely. Even with a >>>>>> 60 second >>>>>> > delay, the lookup fails with some frequency. Am I missing something >>>>>> here? Is >>>>>> > there some Django query cache that I can clear? Or should I just >>>>>> rewrite all >>>>>> > this to just to use a look-back perspective. >>>>>> > >>>>>> > The stack is Django 1.4 connecting to MySQL 5.5.27. Django is >>>>>> handling >>>>>> > 200-1000 requests per second and the database is handling about >>>>>> double that. >>>>>> > >>>>>> > Thanks, >>>>>> > Chris >>>>>> >>>>>> from django import transaction >>>>>> … >>>>>> obj.save() >>>>>> transaction.commit() >>>>>> task.submit(obj.id) >>>>>> >>>>>> You will also need to make sure that gearman is doing things >>>>>> correctly >>>>>> as well. You haven't mentioned what database you are using, but if >>>>>> gearman's DB connection is in a read repeated mode, you can do >>>>>> whatever you like in django but you won't see new data in gearman >>>>>> until gearman's current transaction is committed. >>>>>> >>>>>> Cheers >>>>>> >>>>>> Tom >>>>>> >>>>> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to django-users...@googlegroups.com. >> To post to this group, send email to django...@googlegroups.com. >> Visit this group at http://groups.google.com/group/django-users?hl=en. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> >> -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-users...@googlegroups.com <javascript:>. > To post to this group, send email to django...@googlegroups.com<javascript:> > . > Visit this group at http://groups.google.com/group/django-users?hl=en. > For more options, visit https://groups.google.com/groups/opt_out. > > > > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
