Minor correction: I placed Atomic.__exit__ to verify - the transaction is
commited every time *starting* on the second object (the third stack
printed in the previous post) - it happens at
https://github.com/django/django/blob/master/django/db/transaction.py#L288.
On Sunday, June 23, 2013 7:24:40 PM UTC-4, Yo-Yo Ma wrote:
>
> Hi again Russell,
>
> I did a little digging. I'm not sure, but I may have uncovered the
> problem. A transaction block (using `commit_on_success_unless_managed`) is
> entered and exited during each fixture object loaded, due to the calls to
> the aforementioned method that exist in various model methods (namely,
> `save_base`, in this case). Because of this, the transaction is committed
> immedately after each object is loaded, despite the attempt to wrap
> `commit_on_success_unless_managed` around the context of the `loaddata`
> call in the management command.
>
> The following are the results of my placing print statements (I know
> that's old-school - pdb is just too time consuming) inside
> `commit_on_success_unless_managed`. In each call, I added:
>
> print 'AUTOCOMMIT', connection.autocommit
> print 'IN ATOMIC BLOCK', connection.in_atomic_block
> for frame in inspect.stack():
> print frame[1], frame[3], frame[2]
>
> as well as a print after the stack of whether atomic() was returned or
> _transaction_func() was returned (for easier reading):
>
>
> AUTOCOMMIT False
> IN ATOMIC BLOCK False
>
> django/db/transaction.py commit_on_success_unless_managed 492
> django/core/management/commands/loaddata.py handle 53
> django/core/management/base.py execute 283
> django/core/management/base.py run_from_argv 240
> django/core/management/__init__.py execute 392
> django/core/management/__init__.py execute_from_command_line 399
> manage.py <module> 10
>
> ----RETURNING TRANSACTION FUNC
>
> ===========================================================
>
> AUTOCOMMIT False
> IN ATOMIC BLOCK False
>
> django/db/transaction.py commit_on_success_unless_managed 492
> django/db/models/base.py save_base 573
> django/core/serializers/base.py save 165
> django/core/management/commands/loaddata.py process_dir 225
> django/core/management/commands/loaddata.py load_label 169
> django/core/management/commands/loaddata.py loaddata 102
> django/core/management/commands/loaddata.py handle 54
> django/core/management/base.py execute 283
> django/core/management/base.py run_from_argv 240
> django/core/management/__init__.py execute 392
> django/core/management/__init__.py execute_from_command_line 399
> manage.py <module> 10
>
> ----RETURNING TRANSACTION FUNC
>
> ===========================================================
>
> SAVEPOINT False
> AUTOCOMMIT True
> IN ATOMIC BLOCK False
>
> |||||||||||||||||||||||||||||||||||||||||||||||
> django/db/transaction.py commit_on_success_unless_managed 492
> django/db/models/base.py save_base 573
> django/core/serializers/base.py save 165
> django/core/management/commands/loaddata.py process_dir 225
> django/core/management/commands/loaddata.py load_label 169
> django/core/management/commands/loaddata.py loaddata 102
> django/core/management/commands/loaddata.py handle 54
> django/core/management/base.py execute 283
> django/core/management/base.py run_from_argv 240
> django/core/management/__init__.py execute 392
> django/core/management/__init__.py execute_from_command_line 399
> manage.py <module> 10
>
> ----RETURNING ATOMIC
>
> ===========================================================
>
>
> The remaining calls were exactly like call 3 (including "IN ATOMIC BLOCK
> False", despite the 3rd call having returned `atomic()`). My prima facie
> opinion is that `with atomic()` is needed in `loaddata`, instead of `with
> commit_on_success_unless_managed`, since the latter acts funky when nested
> calls occur (as see in save_base in the stacks printed above). However, the
> issue might be something that needs to be resolved in the
> transitioning-to-atomic code. I don't fully understand all of this yet, but
> it's a start.
>
>
> On Friday, June 21, 2013 4:34:14 PM UTC-4, Yo-Yo Ma wrote:
>>
>> Pardon one typo: I meant `python manage.py loaddata test_data` in my
>> previous post.
>>
>> On Friday, June 21, 2013 4:32:33 PM UTC-4, Yo-Yo Ma wrote:
>>>
>>> Hi Russel,
>>>
>>> Thanks for taking the time to explain that. I tried that same day to
>>> reproduce the issue in a testing env with the simplified models I typed
>>> above, but my hosting provider had some erroneous networking nonsense that
>>> ruined my test after I spent a couple hours setting everything up. I
>>> figured I'm come back to it... and here I am.
>>>
>>> I didn't set up an entire test env and test app this time, just made a
>>> fresh database and ran my apps fixtures on it, but I did test my app again,
>>> using a fresh database without any data. The models and fixtures for which
>>> are as follows (minus most of the decimals, chars, and other non-FK-type
>>> fields, none of which should be related to this problem):
>>>
>>>
>>> # account/models.py
>>> class Account(models.Model):
>>> name = models.CharField(_(u'name'), max_length=255)
>>>
>>>
>>> # orders/models.py
>>> class Order(models.Model):
>>> account = models.ForeignKey('account.Account',
>>> verbose_name=_(u'account'))
>>> number = models.IntegerField(_(u'number'))
>>> bill_address = models.OneToOneField(
>>> 'orders.OrderAddress',
>>> null=True,
>>> on_delete=models.SET_NULL,
>>> related_name='bill_address_order',
>>> verbose_name=_(u'bill to address')
>>> )
>>>
>>> class OrderAddress(models.Model):
>>> account = models.ForeignKey('account.Account',
>>> verbose_name=_(u'account'))
>>> order = models.ForeignKey('orders.Order', verbose_name=_(u'order'))
>>> country = models.CharField(_(u'country'), max_length=2)
>>>
>>>
>>> // orders/fixtures/test_data.json
>>> [
>>> {
>>> "model": "orders.order",
>>> "pk": 1,
>>> "fields": {
>>> "account": 1,
>>> "number": 1,
>>> "bill_address": 1
>>> }
>>> },
>>> {
>>> "model": "orders.orderaddress",
>>> "pk": 1,
>>> "fields": {
>>> "account": 1,
>>> "order": 1,
>>> "country": "US",
>>> }
>>> }
>>> ]
>>>
>>>
>>> (an Account with the primary key of 1 already exists at the time of
>>> ``loaddata``)
>>>
>>>
>>> The error I get with `python manage.py loaddata test_data orders` is:
>>>
>>> django.db.utils.IntegrityError: Problem installing fixture
>>> '/opt/myproject/apps/orders/fixtures/test_data.json': Could not load
>>> orders.Order(pk=1): insert or update on table "orders_order" violates
>>> foreign key constraint "bill_address_id_refs_id_3a4d3fef"
>>> DETAIL: Key (bill_address_id)=(1) is not present in table
>>> "orders_orderaddress".
>>>
>>>
>>> The above fixtures load locally, and they load during test running (with
>>> Postgres) a number of times, but for some reason I get that error when
>>> using `manage.py loaddata ...`.
>>>
>>>
>>> On Sunday, June 16, 2013 7:40:02 PM UTC-4, Russell Keith-Magee wrote:
>>>>
>>>>
>>>> Circular dependencies *shouldn't* be a problem on PostgreSQL because
>>>> all constraints are set DEFERABLE INITIALLY DEFERRED; that means no
>>>> constrain checks should be performed are performed until the transaction
>>>> boundary, so all circular references shouldn't be a problem.
>>>>
>>>> Ticket #3615 exists because MySQL's implementation of DEFERABLE
>>>> INITIALLY DEFERRED under InnoDB is, to use the technical term, "Broken".
>>>> It's unrelated to any problem you may have found in PostgreSQL, because
>>>> PostgreSQL gets the underlying behaviour right.
>>>>
>>>> Beyond that, we need a specific test case to take this any further. As
>>>> it stands, I'm not aware of any problems loading fixtures into PostgreSQL.
>>>> If you are able to construct and provide a set of models (which you have
>>>> done) and simple fixture (which you haven't) that fails reliably, we have
>>>> a
>>>> new bug on our hands, and you should open a ticket with all the details
>>>> you
>>>> can provide. Confirming whether this is a problem with the alpha, or an
>>>> ongoing problem would also be helpful.
>>>>
>>>> Yours,
>>>> Russ Magee %-)
>>>>
>>>> On Mon, Jun 17, 2013 at 6:22 AM, Yo-Yo Ma <[email protected]> wrote:
>>>>
>>>>> There doesn't appear to be a way to load fixtures from JSON (using
>>>>> Postgres - works fine in sqlite3) for the following models:
>>>>>
>>>>>
>>>>> class Collection(models.Model):
>>>>> main_thing = models.OneToOneField(
>>>>> 'things.Thing',
>>>>> null=True,
>>>>> on_delete=models.SET_NULL
>>>>> )
>>>>>
>>>>> class Thing(models.Model):
>>>>> collection = models.ForeignKey(
>>>>> 'collections.Collection'
>>>>> )
>>>>>
>>>>>
>>>>> Here is the exception:
>>>>>
>>>>> Problem installing fixture 'my_fixture.json': Could not load
>>>>> collections.Collection(pk=1): insert or update on table
>>>>> "collections_collection" violates foreign key constraint
>>>>> "main_thing_id_refs_id_3a4d3fef"
>>>>> DETAIL: Key (main_thing_id)=(1) is not present in table
>>>>> "things_thing".
>>>>>
>>>>> I'm not sure if the issue is due to the unique constraint implied by a
>>>>> OneToOneField, or if it's just related to this issue:
>>>>> https://code.djangoproject.com/ticket/3615 (seems like that ticket
>>>>> and related ones have been closed for years, so possibly not related).
>>>>>
>>>>> Any thoughts?
>>>>>
>>>>> Note: I'm using @1.6a1
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Django developers" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to [email protected].
>>>>> To post to this group, send email to [email protected].
>>>>> Visit this group at http://groups.google.com/group/django-developers.
>>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>>>
>>>>>
>>>>>
>>>>
>>>>
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.
