Re: [GENERAL] Where can I get the number of plans that considered by Planner?

2009-10-02 Thread 纪晓曦
Since I also need to consider gego, is this the best way to do it? 2009/9/30 Tom Lane > =?UTF-8?B?57qq5pmT5pum?= writes: > > Where can I add a integer counter to count the plans considered by > planner. > > Well, you could count the number of calls to add_path, but a path is > hardly the same

[GENERAL] [ANN] VTD-XML 2.7

2009-10-02 Thread jimmy Zhang
VTD-XML 2.7 is released and can be downloaded at http://sourceforge.net/projects/vtd-xml/files/ Below is a summary of what are the new features and enhancements. Expanded VTD-XML's Core API * VTDNav: toStringUpperCase, toStringLowerCase, contains(), endsWith(), startsWith() * Extended VTD added

Re: [GENERAL] Limit of bgwriter_lru_maxpages of max. 1000?

2009-10-02 Thread Greg Smith
On Fri, 2 Oct 2009, Scott Marlowe wrote: I found that lowering checkpoint completion target was what helped. Does that seem counter-intuitive to you? Generally, but there are plenty of ways you can get into a state where a short but not immediate checkpoint is better. For example, consider a

Re: [GENERAL] Limit of bgwriter_lru_maxpages of max. 1000?

2009-10-02 Thread Scott Marlowe
On Fri, Oct 2, 2009 at 3:55 PM, Tom Lane wrote: > Scott Marlowe writes: >> The session servers we have at work are a perfect match for this.  By >> increasing checkpoint segments to 100 (or more), timeout to 60 >> minutes, and setting completion target lower (currently 0.25) we have >> reduced ou

Re: [GENERAL] Limit of bgwriter_lru_maxpages of max. 1000?

2009-10-02 Thread Tom Lane
Bruce Momjian writes: > My guess is that having a very long fuzzy checkpoint time means that > when you fsync you are fsync'ing lots of data, both your checkpoint data > and other writes performed by backends. Hmm, could be ... although that would imply that shared_buffers should be kicked up som

Re: [GENERAL] Limit of bgwriter_lru_maxpages of max. 1000?

2009-10-02 Thread Bruce Momjian
Tom Lane wrote: > Scott Marlowe writes: > > The session servers we have at work are a perfect match for this. By > > increasing checkpoint segments to 100 (or more), timeout to 60 > > minutes, and setting completion target lower (currently 0.25) we have > > reduced our IO wait from 10 to 15% to n

Re: [GENERAL] Limit of bgwriter_lru_maxpages of max. 1000?

2009-10-02 Thread Tom Lane
Scott Marlowe writes: > The session servers we have at work are a perfect match for this. By > increasing checkpoint segments to 100 (or more), timeout to 60 > minutes, and setting completion target lower (currently 0.25) we have > reduced our IO wait from 10 to 15% to nearly nothing. These are

Re: [GENERAL] Procedure for feature requests?

2009-10-02 Thread Joshua D. Drake
On Fri, 2009-10-02 at 21:37 +, Tim Landscheidt wrote: > Alvaro Herrera wrote: > > > [...] > >> Suppose the feature request was not a trivial one, but > >> maybe a "DEPENDS ON " clause for "CREATE FUNCTION" > >> to allow PostgreSQL to deny requests to drop a table/view/ > >> function that is

Re: [GENERAL] Boolean storage takes up 1 byte?

2009-10-02 Thread Tom Lane
Simon Riggs writes: > It might be possible to make BOOLEAN NOT NULL use the null bit to > represent the actual data value and then have the column use no > additional bytes, except when we don't store the null bitmap at all. > Just needs people to make it happen cleanly, if that's possible. I rea

Re: [GENERAL] Procedure for feature requests?

2009-10-02 Thread Tim Landscheidt
Sam Mason wrote: >> suppose I thought that PostgreSQL would benefit greatly from >> a "generate_series(DATE, DATE[, INT]) RETURNS DATE" function > 8.4 has a generate_series(timestamp,timestamp,interval) which would seem > to be a bit more flexible than you want. Yes, I know :-). But as "generat

Re: [GENERAL] Procedure for feature requests?

2009-10-02 Thread Tim Landscheidt
Alvaro Herrera wrote: > [...] >> Suppose the feature request was not a trivial one, but >> maybe a "DEPENDS ON " clause for "CREATE FUNCTION" >> to allow PostgreSQL to deny requests to drop a table/view/ >> function that is needed by a function - where would I pro- >> pose that? > On -hackers,

Re: [GENERAL] Vacuumdb Fails: Huge Tuple

2009-10-02 Thread Tom Lane
Teodor Sigaev writes: > ginHeapTupleFastCollect and ginEntryInsert checked tuple's size for > TOAST_INDEX_TARGET, but ginHeapTupleFastCollect checks without one > ItemPointer, > as ginEntryInsert does it. So ginHeapTupleFastCollect could produce a tuple > which 6-bytes larger than allowed by g

Re: [GENERAL] Limit of bgwriter_lru_maxpages of max. 1000?

2009-10-02 Thread Scott Marlowe
On Fri, Oct 2, 2009 at 2:19 PM, Greg Smith wrote: > On Fri, 2 Oct 2009, Gerhard Wiesinger wrote: > >> In my experience flushing I/O as soon as possible is the best solution. > > That what everyone assumes, but detailed benchmarks of PostgreSQL don't > actually support that view given how the datab

Re: [GENERAL] PostgreSQL reads each 8k block - no larger blocks are used - even on sequential scans

2009-10-02 Thread Greg Smith
On Fri, 2 Oct 2009, Gerhard Wiesinger wrote: Larger blocksizes also reduce IOPS (I/Os per second) which might be a critial threshold on storage systems (e.g. Fibre Channel systems). True to some extent, but don't forget that IOPS is always relative to a block size in the first place. If you'

Re: [GENERAL] Limit of bgwriter_lru_maxpages of max. 1000?

2009-10-02 Thread Greg Smith
On Fri, 2 Oct 2009, Gerhard Wiesinger wrote: In my experience flushing I/O as soon as possible is the best solution. That what everyone assumes, but detailed benchmarks of PostgreSQL don't actually support that view given how the database operates. We went through a lot of work in 8.3 relat

Re: [GENERAL] Time Management - Training Seminar in Cape Town

2009-10-02 Thread David Fetter
On Fri, Oct 02, 2009 at 08:32:50PM +0100, Simon Riggs wrote: > > On Wed, 2009-09-30 at 23:48 -0500, Training wrote: > > > A training seminar that will put more time back in your life. > > Attending PostgreSQL training will help you to target your > prospective customers more effectively and sto

Re: [GENERAL] PostgreSQL reads each 8k block - no larger blocks are used - even on sequential scans

2009-10-02 Thread Simon Riggs
On Sun, 2009-09-27 at 18:05 +0200, Gerhard Wiesinger wrote: > So I saw, that even on sequential reads (and also on bitmap heap scan acces) > PostgreSQL uses only 8k blocks. I think that's a major I/O bottleneck. > > A commercial software database vendor solved the problem by reading multiple >

Re: [GENERAL] Boolean storage takes up 1 byte?

2009-10-02 Thread Simon Riggs
On Thu, 2009-10-01 at 12:03 +0100, Sam Mason wrote: > > However, I see that a boolean takes up 1 > > byte of storage, which is 8 bits. Is this due to the fact that the value > > can be null? > > I believe it's more to do with the fact that if you add a boolean column > and then subsequently an

Re: [GENERAL] Time Management - Training Seminar in Cape Town

2009-10-02 Thread Simon Riggs
On Wed, 2009-09-30 at 23:48 -0500, Training wrote: > A training seminar that will put more time back in your life. Attending PostgreSQL training will help you to target your prospective customers more effectively and stop wasting their time in the first place. -- Simon Riggs www.2

Re: [GENERAL] Performance evaluation of PostgreSQL's historic releases

2009-10-02 Thread Simon Riggs
On Tue, 2009-09-29 at 09:36 +0200, György Vilmos wrote: > I've done a benchmark of recent versions of PostgreSQL's last five > major releases to see, how performance has changed during the past > years from version to version. > You can find the article here: > http://suckit.blog.hu/2009/09/26/po

Re: [GENERAL] Limit of bgwriter_lru_maxpages of max. 1000?

2009-10-02 Thread Gerhard Wiesinger
On Fri, 2 Oct 2009, Greg Smith wrote: On Sun, 27 Sep 2009, Gerhard Wiesinger wrote: Lowering bgwriter_delay is possible, but I think overhead is too much and still there is a limit of 800MB/s involved: Stuff written by the background writer turns into largely random I/O. 800MB/s of random w

Re: [GENERAL] PostgreSQL reads each 8k block - no larger blocks are used - even on sequential scans

2009-10-02 Thread Gerhard Wiesinger
On Fri, 2 Oct 2009, Greg Smith wrote: On Sun, 27 Sep 2009, Gerhard Wiesinger wrote: I think this is one of the most critical performance showstopper of PostgreSQL on the I/O side. I wish, this is an easy problem compared to the real important ones that need to be resolved. Situations where

Re: [GENERAL] PostgreSQL reads each 8k block - no larger blocks are used - even on sequential scans

2009-10-02 Thread Gerhard Wiesinger
On Fri, 2 Oct 2009, Greg Smith wrote: On Sun, 27 Sep 2009, Gerhard Wiesinger wrote: I think this is one of the most critical performance showstopper of PostgreSQL on the I/O side. I wish, this is an easy problem compared to the real important ones that need to be resolved. Situations where

Re: [GENERAL] Weird behavior with "sensitive" cursors.

2009-10-02 Thread Alvaro Herrera
Alvaro Herrera escribió: > > This one really works and includes a basic test case. You were right > that the extra Register was bogus :-( I had to expose CopySnapshot, > which I still don't like but ... (I could have added an extra > Unregister somewhere during portal close, but it would have m

Re: [GENERAL] Vacuumdb Fails: Huge Tuple

2009-10-02 Thread Tom Lane
Teodor Sigaev writes: >> Will you apply this, or do you want me to? > I'm not able to provide a good error message in good English :( OK, I'll take care of it later today. regards, tom lane -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make c

Re: [GENERAL] Vacuumdb Fails: Huge Tuple

2009-10-02 Thread Teodor Sigaev
Looks reasonable, although since the error is potentially user-facing I think we should put a bit more effort into the error message (use ereport and make it mention the index name, at least --- is there any other useful information we could give?) Only sizes as it's done in BTree, I suppose. W

Re: [GENERAL] Understanding 'could not read block'

2009-10-02 Thread Joshua D. Drake
On Thu, 2009-09-24 at 05:14 -0700, stevesub wrote: > Hi, > > I have a table of about 693 million rows (80gb) of position data (standard > object,timestamp,position,etc). > > Every time I try to build some statistics by creating a table, such as: > > create table pos_stats1 as > > select id,year,m

Re: [GENERAL] Vacuumdb Fails: Huge Tuple

2009-10-02 Thread Tom Lane
Teodor Sigaev writes: > Patch removes checking of TOAST_INDEX_TARGET and use checking only by > GinMaxItemSize which is greater than TOAST_INDEX_TARGET. All size's check is > now > in GinFormTuple. Looks reasonable, although since the error is potentially user-facing I think we should put a bi

Re: [GENERAL] Vacuumdb Fails: Huge Tuple

2009-10-02 Thread Teodor Sigaev
APseudoUtopia writes: Here's what happened: $ vacuumdb --all --full --analyze --no-password vacuumdb: vacuuming database "postgres" vacuumdb: vacuuming database "web_main" vacuumdb: vacuuming of database "web_main" failed: ERROR: б═huge tuple PostgreSQL 8.4.0 on i386-portbld-freebsd7.2, comp

Re: [GENERAL] Adding modified and creation datetime to system catalogs

2009-10-02 Thread Thom Brown
> > > This has been proposed before, and rejected before, many times. Please > see the archives. > >regards, tom lane > I couldn't find any record of this being mentioned before when I had a look before writing my message, but I'll take your word for it. Thanks Thom

Re: [GENERAL] Procedure for feature requests?

2009-10-02 Thread Sam Mason
On Thu, Oct 01, 2009 at 11:35:25PM +, Tim Landscheidt wrote: > suppose I thought that PostgreSQL would benefit greatly from > a "generate_series(DATE, DATE[, INT]) RETURNS DATE" function 8.4 has a generate_series(timestamp,timestamp,interval) which would seem to be a bit more flexible than you

Re: [GENERAL] Programming interfaces when using MD5 authentication

2009-10-02 Thread Tom Lane
Preston de Guise writes: > Using Perl to program interfaces to PostgreSQL, and had previously > misunderstood how md5 password authentication worked, so I'm now re- > doing it, but struggling to find out how DBD::Pg might be used to > actually authenticate by passing an md5 of the password in

Re: [GENERAL] Adding modified and creation datetime to system catalogs

2009-10-02 Thread Tom Lane
Thom Brown writes: > I was looking around for ways to find out the creation date of a database or > a table, but there doesn't appear to be any functions or available metadata > to provide this information. Unless there's a way I haven't seen yet, does > anyone see any problem with adding a creat

[GENERAL] Adding modified and creation datetime to system catalogs

2009-10-02 Thread Thom Brown
I was looking around for ways to find out the creation date of a database or a table, but there doesn't appear to be any functions or available metadata to provide this information. Unless there's a way I haven't seen yet, does anyone see any problem with adding a creationdatetime and modifieddate

Re: [GENERAL] [pgeu-general] PGDay.EU 2009 - Call for lighting talks

2009-10-02 Thread Hans-Juergen Schoenig
hi dave, here are some suggestions: Artificial intelligence PostgreSQL: We use PostgreSQL to run database driven neural networks as well as classification algorithms for a leading edge application. Our neural networks are running directly inside PostgreSQL. This shows that PostgreSQL is not ju

Re: [GENERAL] Programming interfaces when using MD5 authentication

2009-10-02 Thread Willy-Bas Loos
Hi, I can only answer for the database part. This is on postgres 8.3.8. The passwords can be checked against the table pg_shadow using this algorithm: 'md5'||md5(||) HTH, WBL On Fri, Oct 2, 2009 at 1:48 AM, Preston de Guise wrote: > Hi, > > I apologise in advance if this is considered the wron

Re: [GENERAL] PostgreSQL reads each 8k block - no larger blocks are used - even on sequential scans

2009-10-02 Thread Greg Smith
On Sun, 27 Sep 2009, Sam Mason wrote: The first run of: select count(*) from benchmark; Will cause the "hint" bits to get set and will cause a lot of writing to happen. Subsequent runs will be testing read performance. You just need to be careful of caching effects here. Either stop the da

Re: [GENERAL] PostgreSQL reads each 8k block - no larger blocks are used - even on sequential scans

2009-10-02 Thread Greg Smith
On Sun, 27 Sep 2009, Gerhard Wiesinger wrote: I think this is one of the most critical performance showstopper of PostgreSQL on the I/O side. I wish, this is an easy problem compared to the real important ones that need to be resolved. Situations where the OS is capable of faster sequential

Re: [GENERAL] Limit of bgwriter_lru_maxpages of max. 1000?

2009-10-02 Thread Greg Smith
On Sun, 27 Sep 2009, Gerhard Wiesinger wrote: Lowering bgwriter_delay is possible, but I think overhead is too much and still there is a limit of 800MB/s involved: Stuff written by the background writer turns into largely random I/O. 800MB/s of random writes is so large of a number it's only

[GENERAL] PGDay.EU 2009 - Call for lighting talks

2009-10-02 Thread Dave Page
The PGDay.EU 2009 conference in Paris will include a series of lightning talks on Saturday the 7th November. This is your chance for five minutes of fame and glory, presenting any PostgreSQL-related topic you like! Interested, but not sure what you could speak on? How about: - A PostgreSQL tool