Re: Large Data Sizes, foreign keys, and indexes

2008-05-07 Thread RahulDave
Thanks Doug, I'm going to try to produce a sql copy-able file and get indexes/fkeys later. About debugging, i run the import script from the command line, but do have DEBUG on in my settings file. Does this affect the postgresql debug log? Rahul On May 7, 5:35 pm, Doug B <[EMAIL PROTECTED]> wrot

Re: Large Data Sizes, foreign keys, and indexes

2008-05-07 Thread Doug B
I think you may need to consider looking outside the ORM for imports of that size. I was having trouble with only about 5 million rows, and got a reasonable speed up doing executemany. It was still slow though. I ended up redesigning to eliminate that table, but from what I read in trying to ma

Re: Large Data Sizes, foreign keys, and indexes

2008-05-07 Thread RahulDave
As a follow up, is there a way to have django generate foreign keys as alter table constraints (it does it for some constraints but not others) rather than inline 'references' in create table? On May 6, 11:38 pm, RahulDave <[EMAIL PROTECTED]> wrote: > Hello, > > I have an application which looks

Large Data Sizes, foreign keys, and indexes

2008-05-06 Thread RahulDave
Hello, I have an application which looks lke this Star: id, etc Band:id, etc LightCurve: id, star fkey(Star), band fkey(Band) Postgres is the backend..8.2.3 Every Star basically has multiple light curves (brightness vs time plots) in different spectral bands. Now, there are about 1/2 a billio