Hi Aymeric, It does happen in 1.5, but the error is somehow slightly different (no traceback in 1.5 to find the root cause). I have 2 fields on the order model pointing to the address model. I included only the one in my above example because it was consistently the culprit (likely due to being defined above the other). In 1.5 the *other* field is consistently the culprit:
IntegrityError: Problem installing fixtures: insert or update on table "orders_order" violates foreign key constraint "shipping_rate_id_refs_id_84a732cf" DETAIL: Key (shipping_rate_id)=(2) is not present in table "shipping_shippingrate". The error on 1.6 again is: django.db.utils.IntegrityError: Problem installing fixture '/opt/myproject/apps/orders/fixtures/test_data.json': Could not load orders.Order(pk=1): insert or update on table "orders_order" violates foreign key constraint "bill_address_id_refs_id_3a4d3fef" DETAIL: Key (bill_address_id)=(1) is not present in table "orders_orderaddress". On Tuesday, June 25, 2013 4:59:27 PM UTC-4, Aymeric Augustin wrote: > > Would you mind checking if the bug occurs in Django 1.5? If it doesn't, > it's a regression introduced by the new transaction management in Django > 1.6, and it's a release blocker. > > -- > Aymeric. > > > > On 25 juin 2013, at 22:30, Yo-Yo Ma <[email protected] <javascript:>> > wrote: > > I should actually note, this bug affects all versions of Postgres, and > presumably all other supported RDBMSs as well. > > On Sunday, June 23, 2013 7:35:45 PM UTC-4, Yo-Yo Ma wrote: >> >> Minor correction: I placed Atomic.__exit__ to verify - the transaction is >> commited every time *starting* on the second object (the third stack >> printed in the previous post) - it happens at >> https://github.com/django/django/blob/master/django/db/transaction.py#L288 >> . >> >> >> On Sunday, June 23, 2013 7:24:40 PM UTC-4, Yo-Yo Ma wrote: >>> >>> Hi again Russell, >>> >>> I did a little digging. I'm not sure, but I may have uncovered the >>> problem. A transaction block (using `commit_on_success_unless_managed`) is >>> entered and exited during each fixture object loaded, due to the calls to >>> the aforementioned method that exist in various model methods (namely, >>> `save_base`, in this case). Because of this, the transaction is committed >>> immedately after each object is loaded, despite the attempt to wrap >>> `commit_on_success_unless_managed` around the context of the `loaddata` >>> call in the management command. >>> >>> The following are the results of my placing print statements (I know >>> that's old-school - pdb is just too time consuming) inside >>> `commit_on_success_unless_managed`. In each call, I added: >>> >>> print 'AUTOCOMMIT', connection.autocommit >>> print 'IN ATOMIC BLOCK', connection.in_atomic_block >>> for frame in inspect.stack(): >>> print frame[1], frame[3], frame[2] >>> >>> as well as a print after the stack of whether atomic() was returned or >>> _transaction_func() was returned (for easier reading): >>> >>> >>> AUTOCOMMIT False >>> IN ATOMIC BLOCK False >>> >>> django/db/transaction.py commit_on_success_unless_managed 492 >>> django/core/management/commands/loaddata.py handle 53 >>> django/core/management/base.py execute 283 >>> django/core/management/base.py run_from_argv 240 >>> django/core/management/__init__.py execute 392 >>> django/core/management/__init__.py execute_from_command_line 399 >>> manage.py <module> 10 >>> >>> ----RETURNING TRANSACTION FUNC >>> >>> =========================================================== >>> >>> AUTOCOMMIT False >>> IN ATOMIC BLOCK False >>> >>> django/db/transaction.py commit_on_success_unless_managed 492 >>> django/db/models/base.py save_base 573 >>> django/core/serializers/base.py save 165 >>> django/core/management/commands/loaddata.py process_dir 225 >>> django/core/management/commands/loaddata.py load_label 169 >>> django/core/management/commands/loaddata.py loaddata 102 >>> django/core/management/commands/loaddata.py handle 54 >>> django/core/management/base.py execute 283 >>> django/core/management/base.py run_from_argv 240 >>> django/core/management/__init__.py execute 392 >>> django/core/management/__init__.py execute_from_command_line 399 >>> manage.py <module> 10 >>> >>> ----RETURNING TRANSACTION FUNC >>> >>> =========================================================== >>> >>> SAVEPOINT False >>> AUTOCOMMIT True >>> IN ATOMIC BLOCK False >>> >>> ||||||||||||||||||||||||||||||||||||||||||||||| >>> django/db/transaction.py commit_on_success_unless_managed 492 >>> django/db/models/base.py save_base 573 >>> django/core/serializers/base.py save 165 >>> django/core/management/commands/loaddata.py process_dir 225 >>> django/core/management/commands/loaddata.py load_label 169 >>> django/core/management/commands/loaddata.py loaddata 102 >>> django/core/management/commands/loaddata.py handle 54 >>> django/core/management/base.py execute 283 >>> django/core/management/base.py run_from_argv 240 >>> django/core/management/__init__.py execute 392 >>> django/core/management/__init__.py execute_from_command_line 399 >>> manage.py <module> 10 >>> >>> ----RETURNING ATOMIC >>> >>> =========================================================== >>> >>> >>> The remaining calls were exactly like call 3 (including "IN ATOMIC BLOCK >>> False", despite the 3rd call having returned `atomic()`). My prima facie >>> opinion is that `with atomic()` is needed in `loaddata`, instead of `with >>> commit_on_success_unless_managed`, since the latter acts funky when nested >>> calls occur (as see in save_base in the stacks printed above). However, the >>> issue might be something that needs to be resolved in the >>> transitioning-to-atomic code. I don't fully understand all of this yet, but >>> it's a start. >>> >>> >>> On Friday, June 21, 2013 4:34:14 PM UTC-4, Yo-Yo Ma wrote: >>>> >>>> Pardon one typo: I meant `python manage.py loaddata test_data` in my >>>> previous post. >>>> >>>> On Friday, June 21, 2013 4:32:33 PM UTC-4, Yo-Yo Ma wrote: >>>>> >>>>> Hi Russel, >>>>> >>>>> Thanks for taking the time to explain that. I tried that same day to >>>>> reproduce the issue in a testing env with the simplified models I typed >>>>> above, but my hosting provider had some erroneous networking nonsense >>>>> that >>>>> ruined my test after I spent a couple hours setting everything up. I >>>>> figured I'm come back to it... and here I am. >>>>> >>>>> I didn't set up an entire test env and test app this time, just made a >>>>> fresh database and ran my apps fixtures on it, but I did test my app >>>>> again, >>>>> using a fresh database without any data. The models and fixtures for >>>>> which >>>>> are as follows (minus most of the decimals, chars, and other non-FK-type >>>>> fields, none of which should be related to this problem): >>>>> >>>>> >>>>> # account/models.py >>>>> class Account(models.Model): >>>>> name = models.CharField(_(u'name'), max_length=255) >>>>> >>>>> >>>>> # orders/models.py >>>>> class Order(models.Model): >>>>> account = models.ForeignKey('account.Account', >>>>> verbose_name=_(u'account')) >>>>> number = models.IntegerField(_(u'number')) >>>>> bill_address = models.OneToOneField( >>>>> 'orders.OrderAddress', >>>>> null=True, >>>>> on_delete=models.SET_NULL, >>>>> related_name='bill_address_order', >>>>> verbose_name=_(u'bill to address') >>>>> ) >>>>> >>>>> class OrderAddress(models.Model): >>>>> account = models.ForeignKey('account.Account', >>>>> verbose_name=_(u'account')) >>>>> order = models.ForeignKey('orders.Order', verbose_name=_(u'order')) >>>>> country = models.CharField(_(u'country'), max_length=2) >>>>> >>>>> >>>>> // orders/fixtures/test_data.json >>>>> [ >>>>> { >>>>> "model": "orders.order", >>>>> "pk": 1, >>>>> "fields": { >>>>> "account": 1, >>>>> "number": 1, >>>>> "bill_address": 1 >>>>> } >>>>> }, >>>>> { >>>>> "model": "orders.orderaddress", >>>>> "pk": 1, >>>>> "fields": { >>>>> "account": 1, >>>>> "order": 1, >>>>> "country": "US", >>>>> } >>>>> } >>>>> ] >>>>> >>>>> >>>>> (an Account with the primary key of 1 already exists at the time of >>>>> ``loaddata``) >>>>> >>>>> >>>>> The error I get with `python manage.py loaddata test_data orders` is: >>>>> >>>>> django.db.utils.IntegrityError: Problem installing fixture >>>>> '/opt/myproject/apps/orders/fixtures/test_data.json': Could not load >>>>> orders.Order(pk=1): insert or update on table "orders_order" violates >>>>> foreign key constraint "bill_address_id_refs_id_3a4d3fef" >>>>> DETAIL: Key (bill_address_id)=(1) is not present in table >>>>> "orders_orderaddress". >>>>> >>>>> >>>>> The above fixtures load locally, and they load during test running >>>>> (with Postgres) a number of times, but for some reason I get that error >>>>> when using `manage.py loaddata ...`. >>>>> >>>>> >>>>> On Sunday, June 16, 2013 7:40:02 PM UTC-4, Russell Keith-Magee wrote: >>>>>> >>>>>> >>>>>> Circular dependencies *shouldn't* be a problem on PostgreSQL because >>>>>> all constraints are set DEFERABLE INITIALLY DEFERRED; that means no >>>>>> constrain checks should be performed are performed until the transaction >>>>>> boundary, so all circular references shouldn't be a problem. >>>>>> >>>>>> Ticket #3615 exists because MySQL's implementation of DEFERABLE >>>>>> INITIALLY DEFERRED under InnoDB is, to use the technical term, "Broken". >>>>>> It's unrelated to any problem you may have found in PostgreSQL, because >>>>>> PostgreSQL gets the underlying behaviour right. >>>>>> >>>>>> Beyond that, we need a specific test case to take this any further. >>>>>> As it stands, I'm not aware of any problems loading fixtures into >>>>>> PostgreSQL. If you are able to construct and provide a set of models >>>>>> (which >>>>>> you have done) and simple fixture (which you haven't) that fails >>>>>> reliably, >>>>>> we have a new bug on our hands, and you should open a ticket with all >>>>>> the >>>>>> details you can provide. Confirming whether this is a problem with the >>>>>> alpha, or an ongoing problem would also be helpful. >>>>>> >>>>>> Yours, >>>>>> Russ Magee %-) >>>>>> >>>>>> On Mon, Jun 17, 2013 at 6:22 AM, Yo-Yo Ma <[email protected]>wrote: >>>>>> >>>>>>> There doesn't appear to be a way to load fixtures from JSON (using >>>>>>> Postgres - works fine in sqlite3) for the following models: >>>>>>> >>>>>>> >>>>>>> class Collection(models.Model): >>>>>>> main_thing = models.OneToOneField( >>>>>>> 'things.Thing', >>>>>>> null=True, >>>>>>> on_delete=models.SET_NULL >>>>>>> ) >>>>>>> >>>>>>> class Thing(models.Model): >>>>>>> collection = models.ForeignKey( >>>>>>> 'collections.Collection' >>>>>>> ) >>>>>>> >>>>>>> >>>>>>> Here is the exception: >>>>>>> >>>>>>> Problem installing fixture 'my_fixture.json': Could not load >>>>>>> collections.Collection(pk=1): insert or update on table >>>>>>> "collections_collection" violates foreign key constraint >>>>>>> "main_thing_id_refs_id_3a4d3fef" >>>>>>> DETAIL: Key (main_thing_id)=(1) is not present in table >>>>>>> "things_thing". >>>>>>> >>>>>>> I'm not sure if the issue is due to the unique constraint implied by >>>>>>> a OneToOneField, or if it's just related to this issue: >>>>>>> https://code.djangoproject.com/ticket/3615 (seems like that ticket >>>>>>> and related ones have been closed for years, so possibly not related). >>>>>>> >>>>>>> Any thoughts? >>>>>>> >>>>>>> Note: I'm using @1.6a1 >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Django developers" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To post to this group, send email to [email protected]. >>>>>>> Visit this group at http://groups.google.com/group/django-developers >>>>>>> . >>>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] <javascript:>. > To post to this group, send email to [email protected]<javascript:> > . > Visit this group at http://groups.google.com/group/django-developers. > For more options, visit https://groups.google.com/groups/opt_out. > > > > > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/django-developers. For more options, visit https://groups.google.com/groups/opt_out.
