Re: data migrations for large tables on Postgres

I was thinking of something like that ... but I am not a good enough plpgsql programmer to figure out how to do the chunking in SQL. I needed a way to do the data calculations a thousand rows at a time, rather than all ninety-million in one gulp. So I have Python do the chunking and SQL do th

Re: data migrations for large tables on Postgres

As a newbie - would it be better to use pgadmin (or psql) to make the changes and migrate --fake so that Django would be happy? Johnf On 05/14/2015 02:43 PM, Vernon D. Cole wrote: I have learned the hard way this week that data migrations in django 1.8, as wonderful has they are, do not scale.

data migrations for large tables on Postgres

I have learned the hard way this week that data migrations in django 1.8, as wonderful has they are, do not scale. My test data table is now sitting at about 90,000,000 rows. I was able to add a "null=True" field in an instant, as documented. Then came my attempt to fill it -- I tried using R