Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-17 Thread Tom Lane
Greg Smith writes: > The last time I tried to do this a few years ago I failed miserably and > never came back. I know way more about building software now though, > and just got this to work for the first time. BTW, another thing that should be in the try-try-again category is seeing how clos

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-17 Thread Tom Lane
Robert Haas writes: > ... Maybe someone out there is under the impression > that I get high off of rejecting patches; but the statistics you cite > from the CF app don't exactly support the contention that I'm going > around looking for reasons to reject things, or if I am, I'm doing a > pretty te

Re: [HACKERS] MMAP Buffers

2011-04-17 Thread Radosław Smogura
Robert Haas Monday 18 April 2011 03:06:17 > On Sun, Apr 17, 2011 at 5:32 PM, Radosław Smogura > > wrote: > > Each process has simple "mirror" of shared descriptors. > > > > I "believe" that modifications to buffer content may be only done when > > holding exclusive lock (with some simple except

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-17 Thread Tom Lane
Andrew Dunstan writes: > On 04/17/2011 07:41 PM, Robert Haas wrote: >>> That puts the overall patch acceptance rate at perhaps 75%. >> That someone overstates the acceptance rate, because it ignores the >> patches that people post and immediately get flamed to a well-done >> crisp before adding t

Re: [HACKERS] Evaluation of secondary sort key.

2011-04-17 Thread Jesper Krogh
On 2011-04-09 18:54, Tom Lane wrote: I think that would be a positive disimprovement. The current design guarantees that volatile sort expressions are evaluated exactly once, in the order the rows are read from the data source. There would be no guarantees at all, either as to the number of eva

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-17 Thread Greg Smith
Andrew Dunstan wrote: Now we could certainly make this quite a bit slicker. Apart from anything else, we should change the indent source code tarball so it unpacks into its own directory. Having it unpack into the current directory is ugly and unfriendly. And we should get rid of the "make cle

Re: [HACKERS] MMAP Buffers

2011-04-17 Thread Robert Haas
On Sun, Apr 17, 2011 at 5:32 PM, Radosław Smogura wrote: > Each process has simple "mirror" of shared descriptors. > > I "believe" that modifications to buffer content may be only done when holding > exclusive lock (with some simple exceptions) (+ MVCC), actually I saw only two > things that can c

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-17 Thread Andrew Dunstan
On 04/17/2011 07:41 PM, Robert Haas wrote: That puts the overall patch acceptance rate at perhaps 75%. That someone overstates the acceptance rate, because it ignores the patches that people post and immediately get flamed to a well-done crisp before adding them to the CF app, but there are

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-17 Thread Robert Haas
On Sun, Apr 17, 2011 at 7:13 PM, Tom Lane wrote: > Dan Ports writes: >> ... But I am aware of other cases in which people in the academic community >> have done work that could well be of interest to the Postgres community >> but didn't submit their work here. In part, that was because they did >

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-17 Thread Tom Lane
Dan Ports writes: > ... But I am aware of other cases in which people in the academic community > have done work that could well be of interest to the Postgres community > but didn't submit their work here. In part, that was because they did > not have the time/motivation to get the work into a po

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-17 Thread Dan Ports
On Sat, Apr 16, 2011 at 11:26:34PM -0500, Kevin Grittner wrote: > I have to say, I've been rather mystified by the difficulty > attributed to running pgindent. During work on the SSI patch, I ran > it about once every two weeks on files involved in the patch Well, as a counterpoint: during work o

Re: [HACKERS] MMAP Buffers

2011-04-17 Thread Radosław Smogura
Robert Haas Sunday 17 April 2011 22:01:55 > On Sun, Apr 17, 2011 at 11:48 AM, Tom Lane wrote: > > =?utf-8?q?Rados=C5=82aw_Smogura?= writes: > >> Tom Lane Sunday 17 April 2011 01:35:45 > >> > >>> ... Huh? Are you saying that you ask the kernel to map each individual > >>> shared buffer separat

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-17 Thread Robert Haas
On Sun, Apr 17, 2011 at 12:26 AM, Kevin Grittner wrote: > Now, the other aspect to this whole discussion is that people often > have code they have developed for academic purposes or for their own > use which they want to offer to the community "FWIW", and I think we > sometimes miss an opportunit

Re: [HACKERS] MMAP Buffers

2011-04-17 Thread Andres Freund
On Sunday 17 April 2011 22:09:24 Radosław Smogura wrote: > I only know Phenom has 4096 entries I think and this covers 16MB of > memory. The numbers I cited where intels before and after core2. Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] MMAP Buffers

2011-04-17 Thread Radosław Smogura
Andres Freund Sunday 17 April 2011 20:02:11 > On Sunday 17 April 2011 19:26:31 Radosław Smogura wrote: > > Kernel merges vm_structs. So mappings are compacted. I'm not kernel > > specialist, but skipping memory consumption, for not compacted mappings, > > kernel uses btrees for dealing with TLB,

Re: [HACKERS] ALTER TABLE INHERIT vs collations

2011-04-17 Thread Tom Lane
Peter Eisentraut writes: > On Sat, 2011-04-16 at 18:23 -0400, Tom Lane wrote: >> Right at the moment, ALTER INHERIT doesn't verify that collations >> match in a proposed inheritance child. > It should be prevented, I think. BTW, on looking through the source to find the cause, I see that the rea

Re: [HACKERS] MMAP Buffers

2011-04-17 Thread Robert Haas
On Sun, Apr 17, 2011 at 11:48 AM, Tom Lane wrote: > =?utf-8?q?Rados=C5=82aw_Smogura?= writes: >> Tom Lane Sunday 17 April 2011 01:35:45 >>> ... Huh?  Are you saying that you ask the kernel to map each individual >>> shared buffer separately?  I can't believe that's going to scale to >>> realisti

Re: [HACKERS] MMAP Buffers

2011-04-17 Thread Tom Lane
Joshua Berkus writes: > I, for one, am glad he did this work. We've discussed MMAP in the code off > and on for years, but nobody wanted to do the work to test it. Now someone > has, and we can decide whether it's worth pursuing based on the numbers. Well, the troubling issue is that it's not

Re: [HACKERS] MMAP Buffers

2011-04-17 Thread Joshua Berkus
Robert, > Actually, I'd walk through fire for a 10% performance improvement if > it meant only a *risk* to stability. Depends on the degree of risk. MMAP has the potential to introduce instability into areas of the code which have been completely reliable for years. Adding 20 new coredump cas

Re: [HACKERS] MMAP Buffers

2011-04-17 Thread Andres Freund
On Sunday 17 April 2011 19:26:31 Radosław Smogura wrote: > Kernel merges vm_structs. So mappings are compacted. I'm not kernel > specialist, but skipping memory consumption, for not compacted mappings, > kernel uses btrees for dealing with TLB, so it should not matter if there > is 100 vm_struc

Re: [HACKERS] MMAP Buffers

2011-04-17 Thread Radosław Smogura
Tom Lane Sunday 17 April 2011 17:48:56 > =?utf-8?q?Rados=C5=82aw_Smogura?= writes: > > Tom Lane Sunday 17 April 2011 01:35:45 > > > >> ... Huh? Are you saying that you ask the kernel to map each individual > >> shared buffer separately? I can't believe that's going to scale to > >> realistic

Re: [HACKERS] blah blah set client_encoding segfault

2011-04-17 Thread Tom Lane
I wrote: > Not for me ... Oh ... the *second* time, it fails for me. Doh. Will fix. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] blah blah set client_encoding segfault

2011-04-17 Thread Tom Lane
Peter Eisentraut writes: > With the latest master branch: > initdb -D somewhere --locale=C --encoding=LATIN1 > Then in psql: > set client_encoding to utf8; > causes a segfault in the backend. Not for me ... regards, tom lane -- Sent via pgsql-hackers mailing

The big picture for patch submission (was Re: [HACKERS] MMAP Buffers)

2011-04-17 Thread Tom Lane
Robert Haas writes: > ... But please, everyone feel free to continue bashing me for > wanting a readable patch with some understandable submission notes. What he said. All this obsessing over whether the mmap patch could or should have been run through pgindent is missing the big picture. Namely

Re: [HACKERS] MMAP Buffers

2011-04-17 Thread Tom Lane
=?utf-8?q?Rados=C5=82aw_Smogura?= writes: > Tom Lane Sunday 17 April 2011 01:35:45 >> ... Huh? Are you saying that you ask the kernel to map each individual >> shared buffer separately? I can't believe that's going to scale to >> realistic applications. > No, I do > mrempa(mmap_buff_A, MAP_FI

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-17 Thread Andrew Dunstan
On 04/17/2011 09:51 AM, Andrew Dunstan wrote: On 04/17/2011 04:26 AM, Andrew Dunstan wrote: Now we could certainly make this quite a bit slicker. Apart from anything else, we should change the indent source code tarball so it unpacks into its own directory. Having it unpack into the curr

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-17 Thread Andrew Dunstan
On 04/17/2011 04:26 AM, Andrew Dunstan wrote: Now we could certainly make this quite a bit slicker. Apart from anything else, we should change the indent source code tarball so it unpacks into its own directory. Having it unpack into the current directory is ugly and unfriendly. And we sho

[HACKERS] blah blah set client_encoding segfault

2011-04-17 Thread Peter Eisentraut
[sent this a few days ago, must have gotten lost in "moderation" because of the original subject] With the latest master branch: initdb -D somewhere --locale=C --encoding=LATIN1 Then in psql: set client_encoding to utf8; causes a segfault in the backend. The backtrace: #0 SetClientE

Re: [HACKERS] ALTER TABLE INHERIT vs collations

2011-04-17 Thread Peter Eisentraut
On Sat, 2011-04-16 at 21:52 -0400, Jean-Pierre Pelletier wrote: > One use case for this might be with constraint exclusion where there > would be one > partition per collation (language) with queries against the parent table > always > being for exactly one language. If you really wanted that, yo

Re: [HACKERS] ALTER TABLE INHERIT vs collations

2011-04-17 Thread Peter Eisentraut
On Sat, 2011-04-16 at 18:23 -0400, Tom Lane wrote: > Right at the moment, ALTER INHERIT doesn't verify that collations > match in a proposed inheritance child. It should be prevented, I think. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscrip

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-17 Thread Andrew Dunstan
On 04/17/2011 04:08 AM, Greg Smith wrote: Andrew Dunstan wrote: What makes you think this isn't possible to run pgindent? There are no secret incantations. The first hit newbies find looking for info about pgident is http://blog.hagander.net/archives/185-pgindent-vs-dash.html which sure lo

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-17 Thread Andrew Dunstan
On 04/17/2011 02:16 AM, Jeff Janes wrote: On 4/16/11, Andrew Dunstan wrote: What makes you think this isn't possible to run pgindent? There are no secret incantations. A while ago I spent a few hours trying to run it and gave up. I think it was something about needing some obscure BSD vers

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-17 Thread Greg Smith
Robert Haas wrote: But it turns out that it doesn't really matter. Whitespace or no whitespace, if you don't read the diff before you hit send, it's likely to contain some irrelevant cruft, whether whitespace changes or otherwise. Right. Presuming that pgident will actually solve anything

Re: [HACKERS] MMAP Buffers

2011-04-17 Thread Radosław Smogura
Tom Lane Sunday 17 April 2011 01:35:45 > =?utf-8?q?Rados=C5=82aw_Smogura?= writes: > > No, no, no :) I wanted to do this, but from above reason I skipped it. I > > swap VM pages, I do remap, in place where the shared buffer was I put > > mmaped page, and in place where mmaped page was I put share

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-17 Thread Greg Smith
Andrew Dunstan wrote: What makes you think this isn't possible to run pgindent? There are no secret incantations. The first hit newbies find looking for info about pgident is http://blog.hagander.net/archives/185-pgindent-vs-dash.html which sure looks like secret incantations to me. The docu

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-17 Thread Magnus Hagander
On Apr 17, 2011 8:17 AM, "Jeff Janes" wrote: > > On 4/16/11, Andrew Dunstan wrote: > > > > > > What makes you think this isn't possible to run pgindent? There are no > > secret incantations. > > A while ago I spent a few hours trying to run it and gave up. I think > it was something about needin