Re: Object Lookup after Save

2013-05-29 Thread Martin J. Laubach
Well, that is what repeatable read is supposed to do -- once you do a Foo.objects.all() (or whatever) query, re-running the query will give you the same result. That's why it's called repeatable read. You have to either start a new transaction to break that visibility barrier or not use repea

Re: Object Lookup after Save

2013-05-28 Thread Chris Conover
Interesting. Thanks for the rundown. Looking forward to 1.6. On Tuesday, May 28, 2013 6:40:28 PM UTC-4, akaariai wrote: > > On 29 touko, 01:07, Chris Conover wrote: > > Adding commit_unless_managed() before the get() seems to fix the > problem. > > Looking at it locally, the Gearman worker sta

Re: Object Lookup after Save

2013-05-28 Thread akaariai
On 29 touko, 01:07, Chris Conover wrote: > Adding commit_unless_managed() before the get() seems to fix the problem. > Looking at it locally, the Gearman worker starts a transactions and just > calls commit when the processing is done, over and over, never starting a > new transaction. That combin

Re: Object Lookup after Save

2013-05-28 Thread Chris Conover
Also, thank you Tom, Christian, Christophe, and Anssi for helping me track down this problem. Users are now receiving notifications in a timely manner because of you. On Tuesday, May 28, 2013 6:07:22 PM UTC-4, Chris Conover wrote: > > Adding commit_unless_managed() before the get() seems to fix

Re: Object Lookup after Save

2013-05-28 Thread Chris Conover
Adding commit_unless_managed() before the get() seems to fix the problem. Looking at it locally, the Gearman worker starts a transactions and just calls commit when the processing is done, over and over, never starting a new transaction. That combined with REPEATABLE-READ seems to be the culprit

Re: Object Lookup after Save

2013-05-28 Thread Chris Conover
I bribed the DBA to turn on the general query log briefly (which generated 327k logs in 2 minutes and 2 seconds). Looking at the logs, I see the transaction block for the INSERT. The SELECT is definitely coming after the COMMIT in a different thread. I'll try the commit_unless_managed fix quick

Re: Object Lookup after Save

2013-05-28 Thread akaariai
On 28 touko, 22:20, Chris Conover wrote: > Well, you can inspect the object and see it's primary key. Surely that > means the INSERT is completed? So the workflow goes like this: > > foo = Foo() > foo.save() > foo.pk # new primary key is available > submit_gearman_task(foo.pk) > > Then in the Gear

Re: Object Lookup after Save

2013-05-28 Thread Christophe Pettus
Well, Django works. :) I'd strongly suggest examining the data at the database layer to see if you are getting the interaction with the database you expect; you might try bringing it up in a test environment and see if you can reproduce it there. On May 28, 2013, at 1:45 PM, Chris Conover wrot

Re: Object Lookup after Save

2013-05-28 Thread Chris Conover
No, we are not using the TransactionMiddleware. We are using the following Django middlewares: CommonMiddleware, SessionMiddleware, MessageMiddleware, and XFrameOptionsMiddleware. We have a few custom middlewares which handle stuff for the edge caching. Nothing that produces any writes to MySQL.

Re: Object Lookup after Save

2013-05-28 Thread Christophe Pettus
On May 28, 2013, at 1:26 PM, Chris Conover wrote: > # no middleware that does any writes Are you using the Transaction Middleware? -- -- Christophe Pettus x...@thebuild.com -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe fr

Re: Object Lookup after Save

2013-05-28 Thread Chris Conover
I will enable to general query log tonight once the load tapers off and see what's happening. I must say that the opacity of this transaction stuff is pretty frustrating. I don't understand how there can be an enclosing transaction for such a simple view. The view is pretty much: # no middlewar

Re: Object Lookup after Save

2013-05-28 Thread Christophe Pettus
On May 28, 2013, at 12:20 PM, Chris Conover wrote: > Well, you can inspect the object and see it's primary key. Surely that means > the INSERT is completed? That shows the INSERT is completed, but it doesn't show that any enclosing transaction has committed; that's what the database logs will

Re: Object Lookup after Save

2013-05-28 Thread Chris Conover
Well, you can inspect the object and see it's primary key. Surely that means the INSERT is completed? So the workflow goes like this: foo = Foo() foo.save() foo.pk # new primary key is available submit_gearman_task(foo.pk) Then in the Gearman worker: foo = Foo.objects.get(pk=foo_pk) # this caus

Re: Object Lookup after Save

2013-05-28 Thread Christophe Pettus
I'll admit my MySQL skills are a bit rusty, but can you examine the database logs to confirm that the session is in fact doing the INSERT and a commit before the SELECT? On May 28, 2013, at 10:52, Chris Conover wrote: > It's not really feasible to move this project to PostgreSQL. > > I honest

Re: Object Lookup after Save

2013-05-28 Thread Chris Conover
It's not really feasible to move this project to PostgreSQL. I honestly just don't understand the problem. According to the Django documentation, the ORM functions in autocommit mode by default. In other words, the changes should be committed as soon as the query is complete, right? So after an

Re: Object Lookup after Save

2013-05-28 Thread Christophe Pettus
In the meantime, you can use this: https://github.com/Xof/xact On May 28, 2013, at 10:29, Chris Conover wrote: > That seems to only be available in the dev version of Django (now 1.6 alpha). > > On Wednesday, May 22, 2013 4:43:29 AM UTC-4, Christian Schmitt wrote: >> >> if you want transa

Re: Object Lookup after Save

2013-05-28 Thread Chris Conover
That seems to only be available in the dev version of Django (now 1.6 alpha). On Wednesday, May 22, 2013 4:43:29 AM UTC-4, Christian Schmitt wrote: > > if you want transactions you need to do : > > with transaction.atomic(): > > > like described here: > https://docs.djangoproject.com/en/dev/topi

Re: Object Lookup after Save

2013-05-22 Thread Christian Schmitt
if you want transactions you need to do : with transaction.atomic(): like described here: https://docs.djangoproject.com/en/dev/topics/db/transactions/ Am Mittwoch, 22. Mai 2013 10:40:15 UTC+2 schrieb Christian Schmitt: > > In Django the normal behavior should be that when you do a save() it w

Re: Object Lookup after Save

2013-05-22 Thread Christian Schmitt
In Django the normal behavior should be that when you do a save() it will automatically commit() your query's to the database. so that in obj.save() you should just could access the pk with obj.id after you did a obj.save(). If you want to maybe stop the commit you need to do a obj = obj.save(co

Re: Object Lookup after Save

2013-05-21 Thread Chris Conover
Calling transaction.commit() after object.save results in a TransactionManagementError. I mentioned at the end that I am using MySQL (5.5.27). The issue is not that the Gearman workers are having trouble saving their transactions, it's that they are having trouble looking up the incoming object

Re: Object Lookup after Save

2013-05-21 Thread Tom Evans
On Tue, May 21, 2013 at 4:23 PM, Chris Conover wrote: > Hello, > > I'm having an issue looking up objects immediately after they are saved and > am wondering if anyone has any advice on how to address the problem. > Specifically, I'm saving an object in a view (nothing fancy, a simple > save()) an