Hello,
Qui tacet consentire videtur.
I agree there is still a concurrency problem with the approach I
suggested. INSERT ON CONFLICT DO UPDATE might not work, if the
simultaneously inserted object has for some reason other default values,
that the current object, so that DO UPDATE will overwrite the data. I
am not talking about other backends.
Another idea to start with:
1. Add on_conflict=IGNORE parameter to bulk_create() -
https://code.djangoproject.com/ticket/28668
The problem is that in order to insert a lot of data in a database fast,
a single INSERT must be sent, but it fails if any of the records were
already in the database, so if this could happen, one has to iterate
over the input and do a separate INSERT for each object, but this is
slower than a single INSERT.
However with something like INSERT ... ON CONFLICT DO NOTHING (which
varies in the different RDBMs) the database is instructed to absorb all
the data that is not already there.
2. Afterwards extend bulk_create(..., on_conflict=IGNORE) to detect for
Postgresql which objects were actually added, as described at
https://groups.google.com/forum/#!topic/django-developers/wdHIYdQHO_0 ,
and return only those.
Regards
Дилян
On 10/03/2017 06:00 PM, Aymeric Augustin wrote:
Hello,
Since I haven't seen positive feedback from a committer, I'm not
convinced there's consensus about this change.
Also this doesn't look like a particularly easy topic for a beginner. It
raises the following questions:
- INSERT ON CONFLICT UPDATE would likely be a better option on Postgres
≥ 9.5, wouldn't it?
- what about other databases with built-in backends? third-party backends?
- is this implementation really appropriate at any isolation level?
- what are the consequences for performance?
Best regards,
--
Aymeric.
2017-10-03 17:23 GMT+02:00 Дилян Палаузов <[email protected]
<mailto:[email protected]>>:
Hello Muhereza,
I assume you understand by now Django quite well and are willing to
give something "in return".
Currently QuerySet.get_or_create() consists of two SQL commands:
SELECT + optional INSERT. They cause a concurrent problem, if
another get_or_create() is called between the SQL statements.
With the Postgresql backend it is possible to reduce the queries to
a single one.
Consider this table:
CREATE TABLE t (
id SERIAL PRIMARY KEY,
name VARCHAR(10) NOT NULL UNIQUE,
comment VARCHAR(10));
The following query can do what get_or_create() currently achieves:
WITH
maybe_found AS (SELECT * FROM t WHERE t.name
<http://t.name>='nameD'),
to_be_inserted AS (SELECT 'nameD' as "name", 'comment13' as
"comment"),
just_inserted AS (
INSERT INTO t (name, comment) SELECT * FROM to_be_inserted
WHERE NOT EXISTS(SELECT * FROM maybe_found)
RETURNING *)
SELECT *, FALSE as "created" FROM maybe_found UNION ALL
SELECT *, TRUE AS "created" FROM just_inserted LIMIT 2;
where "to_be_inserted' contains the values for the new object
('default' parameter of get_or_create) and 'nameB' in maybe_found is
the criterion passed to get().
The challenge is to integrate the WITH ... SELECT query in Django.
As guidance I can only suggest looking at the existing code.
Regards
Дилян
On 10/03/2017 10:24 AM, Muhereza Herman wrote:
Hello, anyone reading this please help me out.
am a new developer but i would like to contribute to_*django*_.
please guide me on how to do that. thank you.
--
You received this message because you are subscribed to the
Google Groups "Django developers (Contributions to Django
itself)" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to
[email protected]
<mailto:django-developers%[email protected]>
<mailto:[email protected]
<mailto:django-developers%[email protected]>>.
To post to this group, send email to
[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>.
Visit this group at
https://groups.google.com/group/django-developers
<https://groups.google.com/group/django-developers>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/e6d798ac-4ede-45c4-9f20-ca62f0595131%40googlegroups.com
<https://groups.google.com/d/msgid/django-developers/e6d798ac-4ede-45c4-9f20-ca62f0595131%40googlegroups.com>
<https://groups.google.com/d/msgid/django-developers/e6d798ac-4ede-45c4-9f20-ca62f0595131%40googlegroups.com?utm_medium=email&utm_source=footer
<https://groups.google.com/d/msgid/django-developers/e6d798ac-4ede-45c4-9f20-ca62f0595131%40googlegroups.com?utm_medium=email&utm_source=footer>>.
For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.
--
You received this message because you are subscribed to the Google
Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to [email protected]
<mailto:django-developers%[email protected]>.
To post to this group, send email to
[email protected]
<mailto:[email protected]>.
Visit this group at
https://groups.google.com/group/django-developers
<https://groups.google.com/group/django-developers>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/062e270c-85f6-23a5-64a3-b225ddcff3ab%40aegee.org
<https://groups.google.com/d/msgid/django-developers/062e270c-85f6-23a5-64a3-b225ddcff3ab%40aegee.org>.
For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.
--
You received this message because you are subscribed to the Google
Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to [email protected]
<mailto:[email protected]>.
To post to this group, send email to [email protected]
<mailto:[email protected]>.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/CANE-7mVgudhd0wDAvX9_uLxFUJ89w2Lge6-0rOh_U3hPCc4WnA%40mail.gmail.com
<https://groups.google.com/d/msgid/django-developers/CANE-7mVgudhd0wDAvX9_uLxFUJ89w2Lge6-0rOh_U3hPCc4WnA%40mail.gmail.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Django
developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/9182fb5c-4410-b28d-9e30-8d9b295ea22d%40aegee.org.
For more options, visit https://groups.google.com/d/optout.
