Re: [BUGS] BUG #4919: CREATE USER command slows down system performance

2009-08-31 Thread Alvaro Herrera
Lauris Ulmanis wrote: > When user count in Postgres database reaches up to 500 000 - database > command of creating users 'CREATE USER' slows down to 5-10 seconds per user. A bunch of commits along the direction of eliminating what we thought was the main cause of the slowdown have just concluded

Re: [PERFORM] [BUGS] BUG #4919: CREATE USER command slows down system performance

2009-08-08 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > > Tom Lane wrote: > >> Now that we have SQL-level CONNECT privilege, I wonder just how much > >> functionality would be lost if we got rid of the flat files and told > >> people they had to use CONNECT to do any per-user or per-database > >> access control

Re: [PERFORM] [BUGS] BUG #4919: CREATE USER command slows down system performance

2009-08-08 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> In some sense this is a bootstrap problem: what does it take to get to >> the point of being able to read pg_database and its indexes? That is >> necessarily not dependent on the particular database we want to join. >> Maybe we could solve it by having

Re: [PERFORM] [BUGS] BUG #4919: CREATE USER command slows down system performance

2009-08-08 Thread Alvaro Herrera
Tom Lane wrote: > In some sense this is a bootstrap problem: what does it take to get to > the point of being able to read pg_database and its indexes? That is > necessarily not dependent on the particular database we want to join. > Maybe we could solve it by having the relcache write a "global"

Re: [PERFORM] [BUGS] BUG #4919: CREATE USER command slows down system performance

2009-08-08 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> So on third thought, Alvaro's right: the only real solution here is to >> adopt a more efficient representation of the flat files. Maybe some >> sort of simple hashtable arrangement would work. (Rendering them not so >> flat anymore...) > As long as t

Re: [PERFORM] [BUGS] BUG #4919: CREATE USER command slows down system performance

2009-08-08 Thread Alvaro Herrera
Tom Lane wrote: > Actually, I had forgotten that we were using the pg_database flatfile > for purposes other than authentication checks. In particular, we need > it during backend startup to map from database name to database OID, > without which it's impossible to locate the system catalogs for

Re: [PERFORM] [BUGS] BUG #4919: CREATE USER command slows down system performance

2009-08-08 Thread Tom Lane
I wrote: > Alvaro Herrera writes: >> Tom Lane wrote: >>> ... I don't remember exactly what we do with the >>> flat-file contents. >> Maybe what we need is not to get rid of the flat files, but to speed >> them up. If we're worried about speed in the pg_authid flatfile, and >> come up with a solu

Re: [PERFORM] [BUGS] BUG #4919: CREATE USER command slows down system performance

2009-08-08 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> ... I don't remember exactly what we do with the >> flat-file contents. > Maybe what we need is not to get rid of the flat files, but to speed > them up. If we're worried about speed in the pg_authid flatfile, and > come up with a solution to that prob

Re: [PERFORM] [BUGS] BUG #4919: CREATE USER command slows down system performance

2009-08-08 Thread Alvaro Herrera
Tom Lane wrote: > Well, it's a TO-THINK-ABOUT anyway. I think the appropriate next step > would not be to write code, but to do a detailed investigation of what > would be gained or lost. I don't remember exactly what we do with the > flat-file contents. Maybe what we need is not to get rid of

Re: [PERFORM] [BUGS] BUG #4919: CREATE USER command slows down system performance

2009-08-08 Thread Tom Lane
Bruce Momjian writes: > Tom Lane wrote: >> Now that we have SQL-level CONNECT privilege, I wonder just how much >> functionality would be lost if we got rid of the flat files and told >> people they had to use CONNECT to do any per-user or per-database >> access control. >> >> The main point I ca

Re: [PERFORM] [BUGS] BUG #4919: CREATE USER command slows down system performance

2009-08-07 Thread Bruce Momjian
Tom Lane wrote: > Alvaro Herrera writes: > > toruvinn wrote: > >> I was always wondering, though, why PostgreSQL uses this approach and not > >> its catalogs. > > > It does use the catalog for most things. THe flatfile is used for the > > situations where the catalogs are not yet ready to be re

Re: [PERFORM] [BUGS] BUG #4919: CREATE USER command slows down system performance

2009-07-15 Thread Marko Kreen
On 7/15/09, David Wilson wrote: > On Wed, Jul 15, 2009 at 11:10 AM, Marko Kreen wrote: > > From security standpoint, wasting more cycles on bad passwords is good, > > as it decreases the rate bruteforce password scanning can happen. > > > > And I cannot imagine a scenario where performance on

Re: [PERFORM] [BUGS] BUG #4919: CREATE USER command slows down system performance

2009-07-15 Thread Tom Lane
David Wilson writes: > On Wed, Jul 15, 2009 at 11:10 AM, Marko Kreen wrote: >> From security standpoint, wasting more cycles on bad passwords is good, >> as it decreases the rate bruteforce password scanning can happen. >> >> And I cannot imagine a scenario where performance on invalid logins >>

Re: [PERFORM] [BUGS] BUG #4919: CREATE USER command slows down system performance

2009-07-15 Thread David Wilson
On Wed, Jul 15, 2009 at 11:10 AM, Marko Kreen wrote: > From security standpoint, wasting more cycles on bad passwords is good, > as it decreases the rate bruteforce password scanning can happen. > > And I cannot imagine a scenario where performance on invalid logins > can be relevant.. DoS attack

Re: [PERFORM] [BUGS] BUG #4919: CREATE USER command slows down system performance

2009-07-15 Thread Marko Kreen
On 7/15/09, Tom Lane wrote: > Alvaro Herrera writes: > > > toruvinn wrote: > >> I was always wondering, though, why PostgreSQL uses this approach and not > >> its catalogs. > > > It does use the catalog for most things. THe flatfile is used for the > > situations where the catalogs are not y

Re: [BUGS] BUG #4919: CREATE USER command slows down system performance

2009-07-15 Thread Tom Lane
Alvaro Herrera writes: > toruvinn wrote: >> I was always wondering, though, why PostgreSQL uses this approach and not >> its catalogs. > It does use the catalog for most things. THe flatfile is used for the > situations where the catalogs are not yet ready to be read. Now that we have SQL-leve

Re: [BUGS] BUG #4919: CREATE USER command slows down system performance

2009-07-15 Thread Alvaro Herrera
toruvinn wrote: > On Wed, 15 Jul 2009 16:02:09 +0200, Alvaro Herrera > wrote: >> My bet is on the pg_auth flat file. I doubt we have ever tested the >> behavior of that code with 1 billion users ... > I was always wondering, though, why PostgreSQL uses this approach and not > its catalogs. I

Re: [BUGS] BUG #4919: CREATE USER command slows down system performance

2009-07-15 Thread toruvinn
On Wed, 15 Jul 2009 16:02:09 +0200, Alvaro Herrera wrote: My bet is on the pg_auth flat file. I doubt we have ever tested the behavior of that code with 1 billion users ... I've noticed this behaviour some time ago, on a cluster with 50k+ roles (not sure about the number now). Restoring the

Re: [BUGS] BUG #4919: CREATE USER command slows down system performance

2009-07-15 Thread Alvaro Herrera
Lauris Ulmanis wrote: > Hello again! > > I did test on my local test server > > I created up 500 000 users in function loop very quickly - within 48 > seconds. I did again this script reaching up to 1 billion users - results > was the same - 48 seconds. It is very quickly. > > But problem seems

Re: [BUGS] BUG #4919: CREATE USER command slows down system performance

2009-07-15 Thread Lauris Ulmanis
USER command slows down system performance Lauris Ulmanis wrote: > The following bug has been logged online: > > Bug reference: 4919 > Logged by: Lauris Ulmanis > Email address: lauris.ulma...@mykoob.com > PostgreSQL version: 8.3.7, 8.4.0 > Operating system

Re: [BUGS] BUG #4919: CREATE USER command slows down system performance

2009-07-14 Thread Heikki Linnakangas
Lauris Ulmanis wrote: > The following bug has been logged online: > > Bug reference: 4919 > Logged by: Lauris Ulmanis > Email address: lauris.ulma...@mykoob.com > PostgreSQL version: 8.3.7, 8.4.0 > Operating system: Any > Description:CREATE USER command slows down syst

[BUGS] BUG #4919: CREATE USER command slows down system performance

2009-07-14 Thread Lauris Ulmanis
The following bug has been logged online: Bug reference: 4919 Logged by: Lauris Ulmanis Email address: lauris.ulma...@mykoob.com PostgreSQL version: 8.3.7, 8.4.0 Operating system: Any Description:CREATE USER command slows down system performance Details: When user c