Re: PostgreSQL, queries, speed

2014-06-19 Thread Vladimir Chukharev
On Tuesday, June 17, 2014 3:42:56 AM UTC+3, Lachlan Musicman wrote: > > The db is hosted on the same server, it's not underpowered, and it is > less poorly tuned so much as untuned. > > Untuned is the same as poorly tuned when it comes to PostgreSQL. Or worse. Pg has defaults from last millenniu

Re: PostgreSQL, queries, speed

2014-06-17 Thread Tom Evans
On Tue, Jun 17, 2014 at 11:14 PM, Tom Evans wrote: > You want a multi column index when you want to speed up queries > involving all the columns in that query. Or, depending on database features, queries involving all the columns[0:N] - eg an index on (first_name, last_name, sex) would still opti

Re: PostgreSQL, queries, speed

2014-06-17 Thread Tom Evans
On Tue, Jun 17, 2014 at 10:26 PM, Lachlan Musicman wrote: > tkc, > > Is that the default created when adding db_index=True of is that > something that I would do directly in postgres itself? > > cheers > L. Yep, although obviously that won't work when you want a multi-column index. You want a mul

Re: PostgreSQL, queries, speed

2014-06-17 Thread Lachlan Musicman
tkc, Is that the default created when adding db_index=True of is that something that I would do directly in postgres itself? cheers L. On 17 June 2014 22:50, Tim Chase wrote: > On 2014-06-17 11:04, Russell Keith-Magee wrote: >> On Tue, Jun 17, 2014 at 9:25 AM, Lachlan Musicman wrote: >> > Now r

Re: PostgreSQL, queries, speed

2014-06-17 Thread Tim Chase
On 2014-06-17 11:04, Russell Keith-Magee wrote: > On Tue, Jun 17, 2014 at 9:25 AM, Lachlan Musicman wrote: > > Now reading about indexes I can't believe I've not used them > > previously. > > Indicies are definitely your friend - *especially* on > PostgreSQL. :-) I've found http://use-the-index-l

Re: PostgreSQL, queries, speed

2014-06-17 Thread Lachlan Musicman
I discovered what was going wrong - I was configuring pg9.3 but my db was in pg9.1 I think that's a hangover from when I updated from 12.04 to 14.04 or something. Anyway, thankful for backups! (redface) On 17 June 2014 16:50, Alex Mandel wrote: > If you stick to stock python commands virtualenv s

Re: PostgreSQL, queries, speed

2014-06-16 Thread Alex Mandel
If you stick to stock python commands virtualenv shouldn't be involved at all and you can just point to the system python. Thanks, Alex On 06/16/2014 08:16 PM, Lachlan Musicman wrote: > Thanks Russ, appreciated. > > Out of interest, how do people use plpythonu in postgres when virtualenv'd? > >

Re: PostgreSQL, queries, speed

2014-06-16 Thread Lachlan Musicman
Thanks Russ, appreciated. Out of interest, how do people use plpythonu in postgres when virtualenv'd? I'm seeing a lot of "set the PYTHONPATH envvar in Postgres" answers is this the way? What if I have 3 virtualenvs (dev, stage, prod) on the same machine, each grabbing a different DB (dev, stage

Re: PostgreSQL, queries, speed

2014-06-16 Thread Russell Keith-Magee
On Tue, Jun 17, 2014 at 9:25 AM, Lachlan Musicman wrote: > Thank Glen, appreciated. > > Now reading about indexes I can't believe I've not used them previously. > Indicies are definitely your friend - *especially* on PostgreSQL. :-) > Does the addition of "db_index=True" to a field require a m

Re: PostgreSQL, queries, speed

2014-06-16 Thread Lachlan Musicman
Thank Glen, appreciated. Now reading about indexes I can't believe I've not used them previously. Does the addition of "db_index=True" to a field require a migration, or is it acceptable to just add. I guess that it will require a migration, since it's a DB level change. Or an instruction to the

Re: PostgreSQL, queries, speed

2014-06-16 Thread Glen Jungels
You should definitely be able to accomplish what you are needing with a stored function in PostgreSQL. By sending only a single request and letting the database server do the processing for you, you'll minimize how much work the web server has to do. That said, writing a stored function/procedure

Re: PostgreSQL, queries, speed

2014-06-16 Thread Lachlan Musicman
tkc, You make good points, and I will need to look into better use of Django first - aggregates and the like. I have seen the queries sent, and there are plenty. The problem isn't that we are sending 5000 rows - some parts have hundreds of Buckets, each Bucket needs to be tested that it has qty>

Re: PostgreSQL, queries, speed

2014-06-16 Thread Tim Chase
On 2014-06-17 10:08, Lachlan Musicman wrote: > The problem is that even with a relatively small number of parts > (<5000) and only 4 currencies (USD, AUD, EUR, GBP) we are seeing > page rendering slow down as each of those queries is sent off to be > worked out. A couple things occur to me: - hav