#31749: Unable to find effective method to bulk_update else create
-------------------------------------+-------------------------------------
Reporter: XtremeGood | Owner: nobody
Type: New feature | Status: new
Component: Database layer | Version: 3.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: bulk_update, | Triage Stage:
bulk_create | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Description changed by XtremeGood:
Old description:
> I 'm making this code where on daily basis after preprocessing a csv file
> 100, 000 -200, 000 or even more rows will be read. Each row will get
> converted to a model object. Now the csv says me nothing about whether
> those ids existed in the database or not. I have been said
> "update if key already present else create a new one". And the client has
> superficial demands that it should be done in seconds. I said that it's
> not possible and somehow he is ready to wait for 15 minutes at the max.
> That's why I 'm refraining from using create_or_update or other
> functions. Is there any way to achieve the speed and accomplish the task
> at the same time? There are around 100 columns and the file is being read
> in small chunks because I can't read it whole in the memory. It's
> whopping 140 mbs.
New description:
I 'm making this code where on daily basis after preprocessing a csv file
100, 000 -200, 000 or even more rows will be read. Each row will get
converted to a model object. Now the csv says me nothing about whether
those ids existed in the database or not. I have been said
"update if key already present else create a new one". And the client has
demands that it should be done in seconds. I said that it's not possible
and somehow he is ready to wait for 15 minutes at the max. That's why I 'm
refraining from using create_or_update or other functions. Is there any
way to achieve the speed and accomplish the task at the same time? There
are around 100 columns and the file is being read in small chunks because
I can't read it whole in the memory. It's whopping 140 mbs.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/31749#comment:1>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/068.0875fa5e4539ca6d1d9171367a85f7f3%40djangoproject.com.
