Re: [HACKERS] Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Tom Lane
Greg Stark writes: > On Tue, Mar 22, 2011 at 4:35 AM, Tom Lane wrote: >> Reimplement planner's handling of MIN/MAX aggregate optimization (again). > I'm just curious, Why is this no longer an interesting special case? > --- this is an interesting special case as of 9.1 > -explain (costs off) >

[HACKERS] Re: Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Greg Stark
On Tue, Mar 22, 2011 at 4:35 AM, Tom Lane wrote: > Reimplement planner's handling of MIN/MAX aggregate optimization (again). > I'm just curious, Why is this no longer an interesting special case? --- this is an interesting special case as of 9.1 -explain (costs off) - select min(unique2) from

Re: [HACKERS] How to look at the Expression Trees

2011-03-21 Thread Vaibhav Kaushal
Hi, You said: ExecMakeFunctionResultNoSets is used to evaluate function calls. What are the 'functions' there? Are they the user supplied pl/PGSQL style user functions, the functions handled by fmgr or are they just another C function which make the Expression Evaluator? Regards, Vaibhav

Re: [HACKERS] GSoC 2011 - Mentors? Projects?

2011-03-21 Thread Devrim GÜNDÜZ
On Mon, 2011-03-14 at 16:08 +, Dave Page wrote: > > > > Please count me in as a mentor for a GUI project. It is an open > source > > dashboard for PostgreSQL DBAs, called ADJ Dashboard. > > I can't find any info on that via Google. URL please? Erdinc said that he will setup a web page for th

Re: [HACKERS] really lazy vacuums?

2011-03-21 Thread Greg Stark
On Mon, Mar 21, 2011 at 6:08 PM, Jim Nasby wrote: > Has anyone looked at the overhead of measuring how long IO requests to the > kernel take? If we did that not only could we get an idea of what our IO > workload looked like, we could also figure out whether a block came out of > cache or not.

Re: [HACKERS] psql \dt and table size

2011-03-21 Thread David Fetter
+1 for fixing this behavior in 9.1. -1 for changing in 9.0, as the change in behavior mid-release will cause more confusion than the incomplete accounting does. Cheers, David. On Mon, Mar 21, 2011 at 06:44:51PM +0100, Bernd Helmle wrote: > It stroke me today again, that \dt+ isn't displaying the

Re: [HACKERS] Chinese initdb on Windows

2011-03-21 Thread Tom Lane
Heikki Linnakangas writes: > Any objections to the 2nd attached patch, which adds the mapping of > those locale names on Windows? I think the added initdb message isn't following our style guidelines --- it certainly doesn't match the adjacent existing message. Other than that quibble, ok here.

Re: [HACKERS] Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Tom Lane
I wrote: > Hmm. We could possibly fix this by having planagg.c do a completely > separate planner run for each aggregate, wherein it actually does build > the "equivalent" query > SELECT col FROM tab WHERE existing-quals AND col IS NOT NULL > ORDER BY col ASC/DESC LIMIT 1 > and plan th

Re: Sync Rep and shutdown Re: [HACKERS] Sync Rep v19

2011-03-21 Thread Yeb Havinga
On Mon, Mar 21, 2011 at 7:51 PM, Yeb Havinga wrote: > On 2011-03-21 18:04, Robert Haas wrote: > >> On Mon, Mar 21, 2011 at 12:29 PM, Yeb Havinga >> wrote: >> >>> pgbench -i -s 50 test >>> Two runs of "pgbench -c 10 -M prepared -T 600 test" with 1 sync standby - >>> server configs etc were mailed

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-21 Thread Radosław Smogura
Merlin Moncure Monday 21 March 2011 20:58:16 > On Mon, Mar 21, 2011 at 2:08 PM, Greg Stark wrote: > > On Mon, Mar 21, 2011 at 3:54 PM, Merlin Moncure wrote: > >> Can't you make just one large mapping and lock it in 8k regions? I > >> thought the problem with mmap was not being able to detect ot

Re: [HACKERS] really lazy vacuums?

2011-03-21 Thread Jim Nasby
On Mar 16, 2011, at 7:44 PM, Robert Haas wrote: > It > would be really nice (for this and for other things) if we had some > way of measuring the I/O saturation of the system, so that we could > automatically adjust the aggressiveness of background processes > accordingly. Has anyone looked at the

Re: [HACKERS] Chinese initdb on Windows

2011-03-21 Thread Dave Page
On Mon, Mar 21, 2011 at 7:29 PM, Heikki Linnakangas wrote: > On windows, if you have OS locale set to "Chinese (Simplified, PRC)", initdb > fails: > > X:\>C:\pgsql-install\bin\initdb.exe -D data2 > The files belonging to this database system will be owned by user "Heikki". > This user must also ow

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-21 Thread Merlin Moncure
On Mon, Mar 21, 2011 at 2:08 PM, Greg Stark wrote: > On Mon, Mar 21, 2011 at 3:54 PM, Merlin Moncure wrote: >> Can't you make just one large mapping and lock it in 8k regions? I >> thought the problem with mmap was not being able to detect other >> processes >> (http://www.mail-archive.com/pgsql

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-21 Thread Josh Berkus
> Was it really all that bad? IIRC we replaced ARC with the current clock > sweep due to patent concerns. (Maybe there were performance concerns as > well, I don't remember). Yeah, that was why the patent was frustrating. Performance was poor and we were planning on replacing ARC in 8.2 anyway

Re: [HACKERS] Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Tom Lane
Greg Stark writes: > On Mon, Mar 21, 2011 at 5:40 PM, Tom Lane wrote: >> For general aggregates, you >> have to scan the table anyway.  If an index is useful for that, it'll >> get picked up in the normal planning process. > if I do "SELECT count(col) from tab" with no WHERE clauses on a table >

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-21 Thread Alvaro Herrera
Excerpts from Josh Berkus's message of lun mar 21 13:47:21 -0300 2011: > We already did that, actually, when we implemented ARC: effectively gave > PostgreSQL a 3-level cache. The results were not very good, although > the algorithm could be at fault there. Was it really all that bad? IIRC we r

[HACKERS] Chinese initdb on Windows

2011-03-21 Thread Heikki Linnakangas
On windows, if you have OS locale set to "Chinese (Simplified, PRC)", initdb fails: X:\>C:\pgsql-install\bin\initdb.exe -D data2 The files belonging to this database system will be owned by user "Heikki". This user must also own the server process. The database cluster will be initialized with

Re: [HACKERS] tolower() identifier downcasing versus multibyte encodings

2011-03-21 Thread Francisco Figueiredo Jr.
I just received a feedback from our bug report about this problem and it seems the problem also occurred on a windows machine. http://pgfoundry.org/tracker/index.php?func=detail&aid=1010988&group_id=1000140&atid=590 On Sat, Mar 19, 2011 at 14:13, Marko Kreen wrote: > On Sat, Mar 19, 2011 at 5:

[HACKERS] Re: Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Greg Stark
On Mon, Mar 21, 2011 at 5:40 PM, Tom Lane wrote: > Greg Stark writes: >> So it's a clever hack that we used to allow the partial indexes to be >> used. It relied on the implicit assumption that min(x) and max(x) >> where the only values of x where NULL were both NULL. > >> It would be nice if we

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-21 Thread Greg Stark
On Mon, Mar 21, 2011 at 4:47 PM, Josh Berkus wrote: > You're missing my point ... Postgres already *has* a 2-level cache: > shared_buffers and the FS cache.  Anything we add to that will be adding > levels. I don't think those two levels are interesting -- they don't interact cleverly at all. I

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-21 Thread Greg Stark
On Mon, Mar 21, 2011 at 3:54 PM, Merlin Moncure wrote: > Can't you make just one large mapping and lock it in 8k regions? I > thought the problem with mmap was not being able to detect other > processes > (http://www.mail-archive.com/pgsql-general@postgresql.org/msg122301.html) > compatibility is

Re: Sync Rep and shutdown Re: [HACKERS] Sync Rep v19

2011-03-21 Thread Yeb Havinga
On 2011-03-21 18:04, Robert Haas wrote: On Mon, Mar 21, 2011 at 12:29 PM, Yeb Havinga wrote: pgbench -i -s 50 test Two runs of "pgbench -c 10 -M prepared -T 600 test" with 1 sync standby - server configs etc were mailed upthread. - performance as of commit e148443ddd95cd29edf4cc1de6188eb9cee0

Re: [HACKERS] Missing semicolon in parser's gram.y

2011-03-21 Thread Tom Lane
Gurjeet Singh writes: > Attached is a single-line patch to add a missing semicolon to gram.y; caught > using yyextract. Done some time ago ... http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=a051ef699c3ed1f89088dd6bbc2574f13d0b20eb#patch16 regards, tom la

[HACKERS] psql \dt and table size

2011-03-21 Thread Bernd Helmle
It stroke me today again, that \dt+ isn't displaying the acurate table size for tables, since it uses pg_relation_size() till now. With having pg_table_size() since PostgreSQL 9.0 available, i believe it would be more useful to have the total acquired storage displayed, including implicit objec

Re: [HACKERS] Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Tom Lane
Greg Stark writes: > So it's a clever hack that we used to allow the partial indexes to be > used. It relied on the implicit assumption that min(x) and max(x) > where the only values of x where NULL were both NULL. > It would be nice if we were clever enough to support *any* strict > aggregate us

[HACKERS] Re: Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Greg Stark
So it's a clever hack that we used to allow the partial indexes to be used. It relied on the implicit assumption that min(x) and max(x) where the only values of x where NULL were both NULL. It would be nice if we were clever enough to support *any* strict aggregate using partial indexes on WHERE N

Re: Sync Rep and shutdown Re: [HACKERS] Sync Rep v19

2011-03-21 Thread Robert Haas
On Mon, Mar 21, 2011 at 12:29 PM, Yeb Havinga wrote: > pgbench -i -s 50 test > Two runs of "pgbench -c 10 -M prepared -T 600 test" with 1 sync standby - > server configs etc were mailed upthread. > >> - performance as of commit e148443ddd95cd29edf4cc1de6188eb9cee029c5 > > 1158 and 1306 (avg 1232)

Re: [HACKERS] Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Tom Lane
"Kevin Grittner" writes: > Robert Haas wrote: >> Tom Lane wrote: >>> I don't think that suppressing nulls from an index this way is >>> really very useful. Using a partial index probably eats more >>> planner cycles than you'll save, overall. >> If only 1% of the table has non-NULL values in th

[HACKERS] Missing semicolon in parser's gram.y

2011-03-21 Thread Gurjeet Singh
Attached is a single-line patch to add a missing semicolon to gram.y; caught using yyextract. Regards, -- Gurjeet Singh EnterpriseDB Corporation The Enterprise PostgreSQL Company diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index d32e480..44c4fd6 100644 --- a/src/backend/par

Re: [HACKERS] Rectifying wrong Date outputs

2011-03-21 Thread Robert Haas
ters and >>> so you have to take exactly the specified number of digits, not more. > >> Yeah, I thought about that, but it seems that use case is already >> hopelessly broken in both PostgreSQL and Oracle, so I'm disinclined to >> worry about it. > > Ho

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-21 Thread Josh Berkus
On 3/21/11 3:24 AM, Greg Stark wrote: >> 2-level caches work well for a variety of applications. > > I think 2-level caches with simple heuristics like "pin all the > indexes" is unlikely to be helpful. At least it won't optimize the > average case and I think that's been proven. It might be helpf

Re: [HACKERS] Rectifying wrong Date outputs

2011-03-21 Thread Tom Lane
gits, not more. > Yeah, I thought about that, but it seems that use case is already > hopelessly broken in both PostgreSQL and Oracle, so I'm disinclined to > worry about it. How so? regression=# select to_date('20110321', 'MMDD'); to_date -

Re: Sync Rep and shutdown Re: [HACKERS] Sync Rep v19

2011-03-21 Thread Yeb Havinga
On 2011-03-21 02:05, Robert Haas wrote: On Sun, Mar 20, 2011 at 11:03 AM, Yeb Havinga wrote: On 2011-03-20 05:44, Robert Haas wrote: Hmm, I'm not going to be able to reproduce this here, and my test setup didn't show a clear regression. I can try beating on it some more, but... Any chance yo

Re: [HACKERS] Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Kevin Grittner
Robert Haas wrote: > Tom Lane wrote: >> I don't think that suppressing nulls from an index this way is >> really very useful. Using a partial index probably eats more >> planner cycles than you'll save, overall. > > If only 1% of the table has non-NULL values in that column, maybe > not. We

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-21 Thread Heikki Linnakangas
On 21.03.2011 17:54, Merlin Moncure wrote: Can't you make just one large mapping and lock it in 8k regions? I thought the problem with mmap was not being able to detect other processes (http://www.mail-archive.com/pgsql-general@postgresql.org/msg122301.html) compatibility issues (possibly obsole

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-21 Thread Merlin Moncure
On Mon, Mar 21, 2011 at 5:24 AM, Greg Stark wrote: > On Fri, Mar 18, 2011 at 11:55 PM, Josh Berkus wrote: >>> To take the opposite approach... has anyone looked at having the OS just >>> manage all caching for us? Something like MMAPed shared buffers? Even if we >>> find the issue with large sh

Re: [HACKERS] Rectifying wrong Date outputs

2011-03-21 Thread Robert Haas
On Mon, Mar 21, 2011 at 10:18 AM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Mar 21, 2011 at 9:57 AM, Tom Lane wrote: >>> What I was thinking was that would take either 2 or 4 digits. >>> Whatever you do here, the year will have to be delimited by a non-digit >>> for such cases to be

Re: [HACKERS] Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Robert Haas
On Mon, Mar 21, 2011 at 10:25 AM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Mar 21, 2011 at 7:17 AM, Marti Raudsepp wrote: >>> I know that the Merge Append patch required some changes in the >>> min/max optimization, which is probably the cause. > >> Yeah, I think this is a direct result

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-21 Thread rsmogura
On Mon, 21 Mar 2011 10:24:22 +, Greg Stark wrote: On Fri, Mar 18, 2011 at 11:55 PM, Josh Berkus wrote: To take the opposite approach... has anyone looked at having the OS just manage all caching for us? Something like MMAPed shared buffers? Even if we find the issue with large shared buffe

[HACKERS] When and where do PG invoke PLs module?

2011-03-21 Thread _石头
Hello, Thank you for reading my question! Lately,I‘am reading PostgreSQL’s source code! As I understand,the PLs module(such as pl/pgSQL)allow user-defined functions to be writter in other languages. AS "postgresql-9.0-document.pdf" says :"For a function written in a proceduralla

Re: [HACKERS] Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Tom Lane
Robert Haas writes: > On Mon, Mar 21, 2011 at 7:17 AM, Marti Raudsepp wrote: >> I know that the Merge Append patch required some changes in the >> min/max optimization, which is probably the cause. > Yeah, I think this is a direct result of commit > 034967bdcbb0c7be61d0500955226e1234ec5f04. Yea

Re: [HACKERS] Rectifying wrong Date outputs

2011-03-21 Thread Tom Lane
Robert Haas writes: > On Mon, Mar 21, 2011 at 9:57 AM, Tom Lane wrote: >> What I was thinking was that would take either 2 or 4 digits. >> Whatever you do here, the year will have to be delimited by a non-digit >> for such cases to be parseable. > I was assuming a slightly more general vari

Re: [HACKERS] Rectifying wrong Date outputs

2011-03-21 Thread Robert Haas
On Mon, Mar 21, 2011 at 9:57 AM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Mar 21, 2011 at 6:24 AM, Heikki Linnakangas >> wrote: Having said that, it's not entirely clear to me what sane behavior is here.  Personally I would expect that an n-Ys format spec would consume at

Re: [HACKERS] Rectifying wrong Date outputs

2011-03-21 Thread Tom Lane
Robert Haas writes: > On Mon, Mar 21, 2011 at 6:24 AM, Heikki Linnakangas > wrote: >>> Having said that, it's not entirely clear to me what sane behavior is >>> here.  Personally I would expect that an n-Ys format spec would consume >>> at most n digits from the input.  Otherwise how are you goin

Re: [HACKERS] Rectifying wrong Date outputs

2011-03-21 Thread Robert Haas
On Mon, Mar 21, 2011 at 6:24 AM, Heikki Linnakangas wrote: >>> Having said that, it's not entirely clear to me what sane behavior is >>> here.  Personally I would expect that an n-Ys format spec would consume >>> at most n digits from the input.  Otherwise how are you going to use >>> to_date to p

Re: [HACKERS] Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Robert Haas
On Mon, Mar 21, 2011 at 7:17 AM, Marti Raudsepp wrote: > Hi list, > > When I have fields with lots of null values, I often create indexes > like this: CREATE INDEX foo_i ON foo(i) WHERE i IS NOT NULL; > This saves me some space, as most indexed queries exclude NULLs anyway. > > In PostgreSQL 9.0.3

Re: [HACKERS] How to look at the Expression Trees

2011-03-21 Thread Heikki Linnakangas
On 21.03.2011 14:37, Vaibhav Kaushal wrote: I am already using the postgresql server with -d 4 option and it shows a lot of things. But I am not able to see the Expression State trees. To be precise, debug_print_plan=on prints the expression tree that comes from planner, not the execution tree

Re: [HACKERS] How to look at the Expression Trees

2011-03-21 Thread Vaibhav Kaushal
On Mon, Mar 21, 2011 at 5:47 PM, Heikki Linnakangas < heikki.linnakan...@enterprisedb.com> wrote: > > Yes. There's actually two "trees" involved. The planner produces a tree of > Expr nodes, and ExecInitExpr prepares a tree of ExprState nodes that mirrors > the first tree. The ExprStates contain r

Re: [HACKERS] How to look at the Expression Trees

2011-03-21 Thread Heikki Linnakangas
On 21.03.2011 13:44, Vaibhav Kaushal wrote: Hi all, I have been trying to work on the expression evaluator (trying to alter it just for the seqscan case). I have understood a few things. I wish someone could tell me if I am wrong at some point. As far as I have gone through the code, I think: 1

[HACKERS] How to look at the Expression Trees

2011-03-21 Thread Vaibhav Kaushal
Hi all, I have been trying to work on the expression evaluator (trying to alter it just for the seqscan case). I have understood a few things. I wish someone could tell me if I am wrong at some point. As far as I have gone through the code, I think: 1. Quals are formulated by planner 2. Quals are

[HACKERS] Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Marti Raudsepp
Hi list, When I have fields with lots of null values, I often create indexes like this: CREATE INDEX foo_i ON foo(i) WHERE i IS NOT NULL; This saves me some space, as most indexed queries exclude NULLs anyway. In PostgreSQL 9.0.3, min(i) can successfully use this index: --- marti=# create

Re: [HACKERS] Rectifying wrong Date outputs

2011-03-21 Thread Heikki Linnakangas
On 21.03.2011 07:40, Piyush Newe wrote: On Thu, Mar 17, 2011 at 7:56 PM, Tom Lane wrote: Robert Haas writes: On Thu, Mar 17, 2011 at 9:46 AM, Alvaro Herrera wrote: Keep in mind that the datetime stuff was abandoned by the maintainer some years ago with quite some rough edges. Some of it

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-21 Thread Greg Stark
On Fri, Mar 18, 2011 at 11:55 PM, Josh Berkus wrote: >> To take the opposite approach... has anyone looked at having the OS just >> manage all caching for us? Something like MMAPed shared buffers? Even if we >> find the issue with large shared buffers, we still can't dedicate serious >> amounts

Re: [HACKERS] Allowing multiple concurrent base backups

2011-03-21 Thread Heikki Linnakangas
On 18.03.2011 13:56, Heikki Linnakangas wrote: On 18.03.2011 10:48, Heikki Linnakangas wrote: On 17.03.2011 21:39, Robert Haas wrote: On Mon, Jan 31, 2011 at 10:45 PM, Fujii Masao wrote: On Tue, Feb 1, 2011 at 1:31 AM, Heikki Linnakangas wrote: Hmm, good point. It's harmless, but creating th