Re: ManyToManyField and get_or_create

2012-08-31 Thread Matt Long
Bump?

Almost exactly 4 years later, I've ran into this exact same issue in Django 
1.4. Attempting to use get_or_create through a ManyToMany field results in 
an integrity error if the object already exists but is not yet associated 
with the parent object. To use the OP's example, if a Tag with name "foo" 
exists but is not yet associated with a given Thing instance, using 
.tags.get_or_create(name='foo') will indeed raise an IntegrityError:

foo_tag = Tag(name='foo')
foo_tag.save()

a_thing = Thing(name='a')
a_thing.save() 
a_thing.tags.get_or_create(name='foo')

Traceback (most recent call last):
  File "", line 1, in 
  File 
"/usr/local/lib/python2.7/dist-packages/django/db/models/fields/related.py", 
line 616, in get_or_create
super(ManyRelatedManager, self.db_manager(db)).get_or_create(**kwargs)
  File 
"/usr/local/lib/python2.7/dist-packages/django/db/models/manager.py", line 
134, in get_or_create
return self.get_query_set().get_or_create(**kwargs)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", 
line 449, in get_or_create
obj.save(force_insert=True, using=self.db)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/base.py", 
line 463, in save
self.save_base(using=using, force_insert=force_insert, 
force_update=force_update)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/base.py", 
line 551, in save_base
result = manager._insert([self], fields=fields, return_id=update_pk, 
using=using, raw=raw)
  File 
"/usr/local/lib/python2.7/dist-packages/django/db/models/manager.py", line 
203, in _insert
return insert_query(self.model, objs, fields, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", 
line 1576, in insert_query
return query.get_compiler(using=using).execute_sql(return_id)
  File 
"/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", 
line 910, in execute_sql
cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/util.py", 
line 40, in execute
return self.cursor.execute(sql, params)
  File 
"/usr/local/lib/python2.7/dist-packages/django/db/backends/sqlite3/base.py", 
line 337, in execute
return Database.Cursor.execute(self, query, params)
IntegrityError: column name is not unique

I've traced the problem to being that the ManyRelatedManager includes its 
core_filters in its get_query_set method. This results in the "get" portion 
of get_or_create to only return a hit if the Tag exists and is already 
associated to the calling Thing instance. Given the nature of a 
many-to-many relationship, it should not be a requirement that a Tag 
already be linked to the calling Thing for get_or_create to find it; it 
should be enough that the Tag simply exists. All that should happen in that 
scenario is that the (existing) Tag's relationship with the calling Thing 
be created/saved.

On Monday, September 1, 2008 8:04:00 PM UTC-7, Cap wrote:
>
> I'm having problems using get_or_create with a ManyToManyField and I'm 
> not sure whether the problem is me or Django. 
>
> I have two models: 
>
> class Tag(models.Model): 
> name = models.CharField(max_length=256, unique=True) 
>
> class Thing(models.Model): 
> name = models.CharField(max_length=256) 
> tags = models.ManyToManyField(Tag) 
>
> When I try to add a tag that already exists, as: 
>
> a = Thing(name='a') 
> a.save() 
> a.tags.get_or_create(name='foo') 
>
> I get sqlite3.IntegrityError: column name is not unique. 
>
> But when I do it like so: 
>
> a = Thing(name='a') 
> a.save() 
> foo, created = Tag.objects.get_or_create(name='foo') 
> a.tags.add(foo) 
>
> I have no problems. 
>
> I noticed that http://code.djangoproject.com/ticket/3121 seemed to 
> address something like this, so I pulled the latest Django (8834), but 
> the problems remains. 
>
> Am I doing it wrong? 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-users/-/rHntcOgBMAMJ.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.



Re: performance of delete

2013-04-24 Thread Matt Long
I'm curious how long manually running a single delete query with the 
appropriate where clause would take. Surely there's a way to do that through 
the Django ORM without having to drop into raw SQL?

That being said, there are limits to how long a given query can be, so the ORM 
would have to do some intelligent batching of your IN clause, so maybe it does 
make sense to leave that hairy logic out of the ORM :-) 

-- 
Matt Long
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)


On Wednesday, April 24, 2013 at 18:31, Larry Martell wrote:

> I changed it to delete one row at a time and it's taking 3 minutes.
> 
> On Wed, Apr 24, 2013 at 6:38 PM, Setiaman Lee  wrote:
> > Larry,
> > 
> > 
> > Avoid to use IN clause. I'd say to insert the events you want to delete into
> > the temp table. You can put the session id or user id in temp table to
> > differentiate set of data from other users. Then do the delete by joining
> > the temp table to the Event_Message_ldx table. your sql might look like
> > this:
> > 
> > delete event_message_ldx
> > from temp
> > where event_message_ldx.event_id = temp.event_id
> > and temp.userid = %s
> > 
> > above SQL can be executed in Django using RAW SQL. See the link below.
> > https://docs.djangoproject.com/en/dev/topics/db/sql/
> > 
> > Test the SQL in the MySQL workspace when it works then move the code to the
> > Django.
> > 
> > Rgrds,
> > Setiaman Lee
> > 
> > 
> > 
> > 
> > 
> > On Thu, Apr 25, 2013 at 8:12 AM, Larry Martell 
> > wrote:
> > > 
> > > I have a table that has 2.5 million rows and 9 columns that are all
> > > int except for 2 varchar(255) - i.e. not that big of a table. I am
> > > executing this statement:
> > > 
> > > Event_Message_Idx.objects.filter(event_id__in=event_ids).delete()
> > > 
> > > Where event_ids has around 1,500 items in it. There is an index on
> > > event_id. This statement is taking 1 hour and 5 minutes to run. There
> > > is nothing else hitting the database at that time, and the machine
> > > it's running on is 97% idle and has plenty of free memory. This seems
> > > extremely excessive to me. I would guess it's because of the in
> > > clause. Perhaps this is more of a MySQL question when a django one,
> > > but is there some better way to do a delete like this in django?
> > > 
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "Django users" group.
> > > To unsubscribe from this group and stop receiving emails from it, send an
> > > email to django-users+unsubscr...@googlegroups.com.
> > > To post to this group, send email to django-users@googlegroups.com.
> > > Visit this group at http://groups.google.com/group/django-users?hl=en.
> > > For more options, visit https://groups.google.com/groups/opt_out.
> > > 
> > 
> > 
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Django users" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to django-users+unsubscr...@googlegroups.com.
> > To post to this group, send email to django-users@googlegroups.com.
> > Visit this group at http://groups.google.com/group/django-users?hl=en.
> > For more options, visit https://groups.google.com/groups/opt_out.
> > 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
> 
> 


-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.