Re: [HACKERS] switch UNLOGGED to LOGGED

2011-05-10 Thread Leonardo Francalanci
> Yes, that seems like a very appealing approach. There is plenty of > bit-space available in xinfo, and we could reserve a bit each for > nrels, nsubxacts, and nmsgs, with set meaning that an integer count of > that item is present and clear meaning that the count is omitted from > the struct

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-05-10 Thread Heikki Linnakangas
On 10.05.2011 04:43, Greg Smith wrote: Josh Berkus wrote: As I don't think we can change this, I think the best answer is to tell people "Don't submit a big patch to PostgreSQL until you've done a few small patches first. You'll regret it". When I last did a talk about getting started writing

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-10 Thread Peter Geoghegan
On 10 May 2011 02:58, Fujii Masao wrote: >> Alright. I'm currently working on a proof-of-concept implementation of >> that. In the meantime, any thoughts on how this should meld with the >> existing latch implementation? > > How about making WaitLatch monitor the file descriptor for the pipe > by

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-10 Thread Heikki Linnakangas
On 10.05.2011 11:22, Peter Geoghegan wrote: On 10 May 2011 02:58, Fujii Masao wrote: Alright. I'm currently working on a proof-of-concept implementation of that. In the meantime, any thoughts on how this should meld with the existing latch implementation? How about making WaitLatch monitor th

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-05-10 Thread Pavan Deolasee
On Tue, May 10, 2011 at 1:46 PM, Heikki Linnakangas < heikki.linnakan...@enterprisedb.com> wrote: > On 10.05.2011 04:43, Greg Smith wrote: > >> Josh Berkus wrote: >> >>> As I don't think we can change this, I think the best answer is to >>> tell people >>> "Don't submit a big patch to PostgreSQL u

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-10 Thread Peter Geoghegan
On 10 May 2011 09:45, Heikki Linnakangas wrote: > I think we need to refactor the function into something like: > > #define WL_LATCH_SET    1 > #define WL_SOCKET_READABLE 2 > #define WL_SOCKET_WRITEABLE 4 > #define WL_TIMEOUT      8 > #define WL_POSTMASTER_DEATH 16 While I agree with the need to

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 5:14 AM, Peter Geoghegan wrote: > On 10 May 2011 09:45, Heikki Linnakangas > wrote: > >> I think we need to refactor the function into something like: >> >> #define WL_LATCH_SET    1 >> #define WL_SOCKET_READABLE 2 >> #define WL_SOCKET_WRITEABLE 4 >> #define WL_TIMEOUT    

Re: [HACKERS] switch UNLOGGED to LOGGED

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 3:35 AM, Leonardo Francalanci wrote: >> Yes, that seems like a very appealing approach.   There is plenty of >> bit-space available in xinfo, and we could reserve a bit  each for >> nrels, nsubxacts, and nmsgs, with set meaning that an integer count  of >> that item is pres

Re: [HACKERS] switch UNLOGGED to LOGGED

2011-05-10 Thread Leonardo Francalanci
> I don't think making xinfo shorter will save anything, because > whatever follows it is going to be a 4-byte quantity and therefore > 4-byte aligned. ups, didn't notice it. I'll splitxinfo into: uint16 xinfo; uint16 presentFlags; I guess it helps with the reading? I mean, instead

Re: [HACKERS] crash-safe visibility map, take five

2011-05-10 Thread Robert Haas
On Mon, May 9, 2011 at 10:25 PM, Merlin Moncure wrote: > On Fri, May 6, 2011 at 5:47 PM, Robert Haas wrote: >> On Wed, Mar 30, 2011 at 8:52 AM, Heikki Linnakangas >> wrote: Another question: To address the problem in http://archives.postgresql.org/pgsql-hackers/2010-02/msg02097.ph

Re: [HACKERS] switch UNLOGGED to LOGGED

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 8:03 AM, Leonardo Francalanci wrote: >> I don't  think making xinfo shorter will save anything, because >> whatever follows it is  going to be a 4-byte quantity and therefore >> 4-byte aligned. > > > ups, didn't notice it. > > I'll split    xinfo into: > > uint16   xinfo; >

Re: [HACKERS] "stored procedures" - use cases?

2011-05-10 Thread Robert Haas
On Mon, May 9, 2011 at 11:58 PM, Pavel Stehule wrote: > no - you are little bit confused :). CALL and function execution > shares nothing. There is significant differences between function and > procedure. Function is called only from executor - from some plan, and > you have to know a structure o

Re: [HACKERS] "stored procedures" - use cases?

2011-05-10 Thread Pavel Stehule
2011/5/10 Robert Haas : > On Mon, May 9, 2011 at 11:58 PM, Pavel Stehule > wrote: >> no - you are little bit confused :). CALL and function execution >> shares nothing. There is significant differences between function and >> procedure. Function is called only from executor - from some plan, and

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Robert Haas
On Mon, May 9, 2011 at 10:36 PM, Merlin Moncure wrote: >> 1. The visibility map needs to be crash-safe.  The basic idea of >> index-only scans is that, instead of checking the heap to find out >> whether each tuple is visible, we first check the visibility map.  If >> the visibility map bit is set

Re: [HACKERS] crash-safe visibility map, take five

2011-05-10 Thread Merlin Moncure
On Tue, May 10, 2011 at 7:48 AM, Robert Haas wrote: > I thought I'd explained it fairly thoroughly in the comments, but > evidently not.  Suggestions for improvement are welcome. ok. that clears it up nicely. > Here goes in more detail: Every time we insert, update, or delete a > tuple in a par

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Merlin Moncure
On Tue, May 10, 2011 at 8:22 AM, Robert Haas wrote: > On Mon, May 9, 2011 at 10:36 PM, Merlin Moncure wrote: >>> 1. The visibility map needs to be crash-safe.  The basic idea of >>> index-only scans is that, instead of checking the heap to find out >>> whether each tuple is visible, we first chec

Re: [HACKERS] crash-safe visibility map, take five

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 9:45 AM, Merlin Moncure wrote: > I see: here's a comment that was throwing me off: > +       /* > +        * If we didn't get the lock and it turns out we need it, we'll have > to > +        * unlock and re-lock, to avoid holding the buffer lock across an I/O. > +        *

[HACKERS] ts_rank

2011-05-10 Thread Mark
Could somebody explain me on which methods is based ts_rank and how it works? I would appreciate some articles, if exist. Thanks a lot for reply. Mark -- View this message in context: http://postgresql.1045698.n5.nabble.com/ts-rank-tp4384120p4384120.html Sent from the PostgreSQL - hackers mailin

[HACKERS] [v9.2] Leaky view and RLS

2011-05-10 Thread Kohei Kaigai
I'd like to summarize expected issues corresponding to leaky-view and RLS towards v9.2, and PGcon2011/Developer Meeting. We already made consensus the leaky-view is a problem to be fixed previous to the row-level security feature. We know several ways to leak/infer contents of tuples to be invisi

[HACKERS] collateral benefits of a crash-safe visibility map

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 9:59 AM, Merlin Moncure wrote: > no, that wasn't my intent at all, except in the sense of wondering if > a crash-safe visibility map provides a route of displacing a lot of > hint bit i/o and by extension, making alternative approaches of doing > that, including mine, a lot

Re: [HACKERS] improvements to pgtune

2011-05-10 Thread Bruce Momjian
FYI, I can help if you need javascript assistance. --- Greg Smith wrote: > Shiv wrote: > > So my exams are over now and am fully committed to the project in > > terms of time. I have started compiling a sort of personal to

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Cédric Villemain
2011/5/10 Robert Haas : > So, what do we need in order to find our way to index-only scans? > > 3. Statistics.  I believe that in order to accurately estimate the > cost of an index-only scan, we're going to need to know the fraction > of tuples that are on pages whose visibility map bits are set.

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 10:58 AM, Cédric Villemain wrote: > ANALYZE can do the stats job for 'free' on the pages it collects > anyway. So that looks like a good idea. > I believe the really lazy vacuum is another topic; even if it will > improve the performance of the index only scan to have table

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Cédric Villemain
2011/5/10 Robert Haas : > On Tue, May 10, 2011 at 10:58 AM, Cédric Villemain > wrote: >> ANALYZE can do the stats job for 'free' on the pages it collects >> anyway. So that looks like a good idea. >> I believe the really lazy vacuum is another topic; even if it will >> improve the performance of t

Re: [HACKERS] crash-safe visibility map, take five

2011-05-10 Thread Jesper Krogh
On 2011-05-10 14:48, Robert Haas wrote: We could avoid all of this complexity - and the possibility of pinning the visibility map page needlessly - by locking the heap buffer first and then pinning the visibility map page if the heap page is all-visible. However, that would involve holding the l

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Simon Riggs
On Tue, May 10, 2011 at 3:25 AM, Robert Haas wrote: > So, what do we need in order to find our way to index-only scans? > > 1. The visibility map needs to be crash-safe.  The basic idea of > index-only scans is that, instead of checking the heap to find out > whether each tuple is visible, we fir

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Cédric Villemain
>> The temptation is high to estimate the cost of an "index_scan(only) + >> ordered(by ctid) table pages fetch if heap required". (this is what I >> understood from heikki suggestion 3-4. and it makes sense). It may be >> easier to implement both at once but I didn't find the branch in the >> Heikk

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-10 Thread Simon Riggs
On Tue, May 10, 2011 at 12:45 PM, Robert Haas wrote: > On Tue, May 10, 2011 at 5:14 AM, Peter Geoghegan > wrote: >> On 10 May 2011 09:45, Heikki Linnakangas >> wrote: >> >>> I think we need to refactor the function into something like: >>> >>> #define WL_LATCH_SET    1 >>> #define WL_SOCKET_REA

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Kevin Grittner
Simon Riggs wrote: > This topic has been discussed many times, yet I have never seen an > assessment that explains WHY we would want to do index-only scans. In databases with this feature, it's not too unusual for a query which uses just an index to run one or more orders of magnitude faster t

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-10 Thread Tom Lane
Simon Riggs writes: > I've got a feeling that things will go easier if we have a separate > connection for the feedback channel. > Yes, two connections, one in either direction. > That would make everything simple, nice one way connections. It would > also mean we could stream at higher data rat

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-10 Thread Heikki Linnakangas
On 10.05.2011 14:39, Peter Geoghegan wrote: Attached is win32 implementation of the "named pipe trick". It consists of a Visual Studio 2008 solution that contains two projects, named_pipe_trick (which represents the postmaster) and auxiliary_backend (which represents each auxiliary process). I s

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 11:27 AM, Cédric Villemain wrote: > 2011/5/10 Robert Haas : >> On Tue, May 10, 2011 at 10:58 AM, Cédric Villemain >> wrote: >>> ANALYZE can do the stats job for 'free' on the pages it collects >>> anyway. So that looks like a good idea. >>> I believe the really lazy vacuum

Re: [HACKERS] crash-safe visibility map, take five

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 11:34 AM, Jesper Krogh wrote: > On 2011-05-10 14:48, Robert Haas wrote: >> >> We could avoid all of this complexity - and the possibility of pinning >> the visibility map page needlessly - by locking the heap buffer first >> and then pinning the visibility map page if the h

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Tom Lane
"Kevin Grittner" writes: > Simon Riggs wrote: >> This topic has been discussed many times, yet I have never seen an >> assessment that explains WHY we would want to do index-only scans. > In databases with this feature, it's not too unusual for a query > which uses just an index to run one or m

Re: [HACKERS] collateral benefits of a crash-safe visibility map

2011-05-10 Thread Simon Riggs
On Tue, May 10, 2011 at 3:47 PM, Robert Haas wrote: > To address the first problem, what we've talked about doing is > something along the line of freezing the tuples at the time we mark > the page all-visible, so we don't have to go back and do it again > later.  Unfortunately, it's not quite th

Re: [HACKERS] hint bit cache v5

2011-05-10 Thread Simon Riggs
On Mon, May 9, 2011 at 5:12 PM, Merlin Moncure wrote: > I'd like to know if this is a strategy that merits further work...If > anybody has time/interest that is.  It's getting close to the point > where I can just post it to the commit fest for review.  In > particular, I'm concerned if Tom's ear

Re: [HACKERS] crash-safe visibility map, take five

2011-05-10 Thread Tom Lane
Robert Haas writes: > On Tue, May 10, 2011 at 11:34 AM, Jesper Krogh wrote: >> Or what is the downside for keeping it across IO? Will it block other >> processes trying to read it? > Heikki might be in a better position to comment on that than I am, > since he wrote the existing code. But I thi

Re: [HACKERS] collateral benefits of a crash-safe visibility map

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 12:57 PM, Simon Riggs wrote: > Hmmm, do we really need to WAL log freezing? > > Can we break down freezing into a 2 stage process, so that we can have > first stage as a lossy operation and a second stage that is WAL > logged? That might solve the relfrozenxid problem - se

Re: [HACKERS] crash-safe visibility map, take five

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 1:00 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, May 10, 2011 at 11:34 AM, Jesper Krogh wrote: >>> Or what is the downside for keeping it across IO? Will it block other >>> processes trying to read it? > >> Heikki might be in a better position to comment on that

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 12:53 PM, Tom Lane wrote: > "Kevin Grittner" writes: >> Simon Riggs wrote: >>> This topic has been discussed many times, yet I have never seen an >>> assessment that explains WHY we would want to do index-only scans. > >> In databases with this feature, it's not too unusu

Re: [HACKERS] Collation mega-cleanups

2011-05-10 Thread Ross J. Reedstrom
On Mon, May 09, 2011 at 03:57:12PM -0400, Robert Haas wrote: > On Mon, May 9, 2011 at 2:58 PM, Bruce Momjian wrote: > > Tom this collation stuff has seen more post-feature-commit cleanups than > > I think any patch I remember.  Is there anything we can learn from this? > > How about "don't commit

Re: [HACKERS] collateral benefits of a crash-safe visibility map

2011-05-10 Thread Tom Lane
Robert Haas writes: > On Tue, May 10, 2011 at 12:57 PM, Simon Riggs wrote: >> Hmmm, do we really need to WAL log freezing? > That might solve the relfrozenxid problem - set the bits in the heap, > sync the heap, then update relfrozenxid once the heap is guaranteed > safely on disk - but it again

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Simon Riggs
On Tue, May 10, 2011 at 5:17 PM, Kevin Grittner wrote: > Simon Riggs wrote: > >> This topic has been discussed many times, yet I have never seen an >> assessment that explains WHY we would want to do index-only scans. > > In databases with this feature, it's not too unusual for a query > which us

Re: [HACKERS] Collation mega-cleanups

2011-05-10 Thread Andres Freund
On Tuesday, May 10, 2011 07:08:23 PM Ross J. Reedstrom wrote: > On Mon, May 09, 2011 at 03:57:12PM -0400, Robert Haas wrote: > > On Mon, May 9, 2011 at 2:58 PM, Bruce Momjian wrote: > > > Tom this collation stuff has seen more post-feature-commit cleanups > > > than I think any patch I remember.

Re: [HACKERS] collateral benefits of a crash-safe visibility map

2011-05-10 Thread Heikki Linnakangas
On 10.05.2011 17:47, Robert Haas wrote: On Tue, May 10, 2011 at 9:59 AM, Merlin Moncure wrote: no, that wasn't my intent at all, except in the sense of wondering if a crash-safe visibility map provides a route of displacing a lot of hint bit i/o and by extension, making alternative approaches o

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Kevin Grittner
Tom Lane wrote: > "Kevin Grittner" writes: >> Simon Riggs wrote: >>> This topic has been discussed many times, yet I have never seen >>> an assessment that explains WHY we would want to do index-only >>> scans. > >> In databases with this feature, it's not too unusual for a query >> which uses

Re: [HACKERS] hint bit cache v5

2011-05-10 Thread Merlin Moncure
On Tue, May 10, 2011 at 11:59 AM, Simon Riggs wrote: > On Mon, May 9, 2011 at 5:12 PM, Merlin Moncure wrote: > >> I'd like to know if this is a strategy that merits further work...If >> anybody has time/interest that is.  It's getting close to the point >> where I can just post it to the commit f

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-05-10 Thread Greg Smith
Heikki Linnakangas wrote: Well, my first patch was two-phase commit. And I had never even used PostgreSQL before I dived into the source tree and started to work on that Well, everyone knows you're awesome. A small percentage of the people who write patches end up having the combination of ba

Re: [HACKERS] collateral benefits of a crash-safe visibility map

2011-05-10 Thread Merlin Moncure
On Tue, May 10, 2011 at 9:47 AM, Robert Haas wrote: > On Tue, May 10, 2011 at 9:59 AM, Merlin Moncure wrote: >> no, that wasn't my intent at all, except in the sense of wondering if >> a crash-safe visibility map provides a route of displacing a lot of >> hint bit i/o and by extension, making alt

Re: [HACKERS] 4.1beta1: ANYARRAY disallowed for DOMAIN types which happen to be arrays

2011-05-10 Thread J. Greg Davidson
Given: CREATE DOMAIN int_array AS int[]; The operator [] works fine in 4.1beta1: SELECT (ARRAY[1,2,3]::int_array)[1]; proving that int_array is an array type with element type int. It is inconsistent that other array functions and operators don't work. On Mon, 2011-05-09 at 23:32 -0400, T

Re: [HACKERS] Collation mega-cleanups

2011-05-10 Thread Ross J. Reedstrom
On Tue, May 10, 2011 at 07:21:16PM +0200, Andres Freund wrote: > On Tuesday, May 10, 2011 07:08:23 PM Ross J. Reedstrom wrote: > > On Mon, May 09, 2011 at 03:57:12PM -0400, Robert Haas wrote: > > > On Mon, May 9, 2011 at 2:58 PM, Bruce Momjian wrote: > > > > Tom this collation stuff has seen more

Re: [HACKERS] 4.1beta1: ANYARRAY disallowed for DOMAIN types which happen to be arrays

2011-05-10 Thread Robert Haas
On Mon, May 9, 2011 at 11:32 PM, Tom Lane wrote: > "J. Greg Davidson" writes: >>       * Tighten casting checks for domains based on arrays (Tom Lane) > >>         When a domain is based on an array type,..., such a domain type >>         is no longer allowed to match an anyarray parameter of a >

Re: [HACKERS] collateral benefits of a crash-safe visibility map

2011-05-10 Thread Simon Riggs
On Tue, May 10, 2011 at 6:02 PM, Robert Haas wrote: > On Tue, May 10, 2011 at 12:57 PM, Simon Riggs wrote: >> Hmmm, do we really need to WAL log freezing? >> >> Can we break down freezing into a 2 stage process, so that we can have >> first stage as a lossy operation and a second stage that is WA

Re: [HACKERS] collateral benefits of a crash-safe visibility map

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 1:49 PM, Simon Riggs wrote: > On Tue, May 10, 2011 at 6:02 PM, Robert Haas wrote: >> On Tue, May 10, 2011 at 12:57 PM, Simon Riggs wrote: >>> Hmmm, do we really need to WAL log freezing? >>> >>> Can we break down freezing into a 2 stage process, so that we can have >>> fi

Re: [HACKERS] 4.1beta1: ANYARRAY disallowed for DOMAIN types which happen to be arrays

2011-05-10 Thread Tom Lane
Robert Haas writes: > On Mon, May 9, 2011 at 11:32 PM, Tom Lane wrote: >> So we basically had three alternatives to make it better: >>* downcast to the array type, which would possibly silently >> break applications that were relying on the function result >> being consi

Re: [HACKERS] 4.1beta1: ANYARRAY disallowed for DOMAIN types which happen to be arrays

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 1:52 PM, Tom Lane wrote: > Robert Haas writes: >> On Mon, May 9, 2011 at 11:32 PM, Tom Lane wrote: >>> So we basically had three alternatives to make it better: >>>        * downcast to the array type, which would possibly silently >>>          break applications that wer

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-05-10 Thread Josh Berkus
All, > Part of the trouble is in the question. Having a patch rejected is not > really a problem; it's something you should learn from. I know it can be > annoying. I get annoyed when it happens to me too. But I try to get over > it as quickly as possible, and either fix the patch, or find another

Re: [HACKERS] collateral benefits of a crash-safe visibility map

2011-05-10 Thread Simon Riggs
On Tue, May 10, 2011 at 6:08 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, May 10, 2011 at 12:57 PM, Simon Riggs wrote: >>> Hmmm, do we really need to WAL log freezing? > >> That might solve the relfrozenxid problem - set the bits in the heap, >> sync the heap, then update relfrozenxid on

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-10 Thread Peter Geoghegan
On 10 May 2011 17:43, Heikki Linnakangas wrote: > It should be an anonymous pipe that's inherited by the child process by > rather than a named pipe. Otherwise seems fine to me, as far as this proof > of concept program goes. Alright, thanks. I'll use an anonymous pipe in the patch itself. --

Re: [HACKERS] Collation mega-cleanups

2011-05-10 Thread Tom Lane
"Ross J. Reedstrom" writes: > So perhaps it was more of the "This code is less ready than I thought > it was, but now that I've spent the time understanding it and the > problem, the shortest way out is forward". Yeah, exactly. By the time I really understood how incomplete the collation patch w

Re: [HACKERS] Server Programming Interface underspecified in 4.1beta1

2011-05-10 Thread Kevin Grittner
"J. Greg Davidson" wrote: > I would like to be able to program to a C or C++ SPI > which is clean, complete and type-safe. I am good at > reading API documentation in C or C++ and would be happy > to review any proposed improvements. I want to second Andrew's post, and emphasize that such sug

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Greg Stark
On Tue, May 10, 2011 at 5:53 PM, Tom Lane wrote: > It's already the case that we'll flip over to a bitmap indexscan, > and thus get rid of most/all of the "random" page accesses, in > situations where this is likely to be a big win.  Pointing to the > performance difference in databases that don't

Re: [HACKERS] 4.1beta1: ANYARRAY disallowed for DOMAIN types which happen to be arrays

2011-05-10 Thread Tom Lane
Robert Haas writes: > On Tue, May 10, 2011 at 1:52 PM, Tom Lane wrote: >> Robert Haas writes: >>> On Mon, May 9, 2011 at 11:32 PM, Tom Lane wrote: So we basically had three alternatives to make it better: * downcast to the array type, which would possibly silently

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Simon Riggs
On Tue, May 10, 2011 at 6:25 PM, Kevin Grittner wrote: >> ... but I share Simon's desire to see some proof before anything >> gets committed. > > And we agree there.  In fact, I can't think of anyone in the > community who doesn't want to see that for *any* purported > performance enhancement. I

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-05-10 Thread Greg Stark
On Tue, May 10, 2011 at 6:54 PM, Josh Berkus wrote: > Of course, there are always idiots who won't learn anything no matter > how good our process is.  But if the whole submission process is > perceived to be fair and understandible, those will be a tiny minority. The thing is, I think things are

Re: [HACKERS] Backpatching of "Teach the regular expression functions to do case-insensitive matching"

2011-05-10 Thread Peter Eisentraut
On mån, 2011-05-09 at 10:56 -0400, Robert Haas wrote: > I'm just shooting from the hip here, but maybe we could have a > separate (probably smaller) set of tests that are only designed to > work in a limited range of locales and/or encodings. I'm really > pleased that we now have the src/test/isol

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-05-10 Thread Josh Berkus
> The thing is, I think things are much better now than they were three > or four years ago. Oh, no question. If you read above in this thread, I'm not really proposing a change in the current process, just documenting the current process. Right now there's a gap between how sumbitters expect t

Re: [HACKERS] Backpatching of "Teach the regular expression functions to do case-insensitive matching"

2011-05-10 Thread Peter Eisentraut
On mån, 2011-05-09 at 12:42 -0400, Tom Lane wrote: > The problem we'd have is that there's no way (at present) to make such > a test pass on every platform. Windows has its own set of locale names > (which initdb fails to install as collations anyway) and we also have > the problem that OS X can b

Re: [HACKERS] Backpatching of "Teach the regular expression functions to do case-insensitive matching"

2011-05-10 Thread Tom Lane
Peter Eisentraut writes: > On mån, 2011-05-09 at 12:42 -0400, Tom Lane wrote: >> The problem we'd have is that there's no way (at present) to make such >> a test pass on every platform. Windows has its own set of locale names >> (which initdb fails to install as collations anyway) and we also ha

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Kevin Grittner
Simon Riggs wrote: > Kevin Grittner wrote: > >>> ... but I share Simon's desire to see some proof before anything >>> gets committed. >> >> And we agree there. In fact, I can't think of anyone in the >> community who doesn't want to see that for *any* purported >> performance enhancement. > >

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 3:09 PM, Greg Stark wrote: > The thing is, I think things are much better now than they were three > or four years ago. At the time the project had grown much faster than > the existing stable of developers and the rate at which patches were > being submitted was much great

Re: [HACKERS] Backpatching of "Teach the regular expression functions to do case-insensitive matching"

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 3:09 PM, Peter Eisentraut wrote: > On mån, 2011-05-09 at 10:56 -0400, Robert Haas wrote: >> I'm just shooting from the hip here, but maybe we could have a >> separate (probably smaller) set of tests that are only designed to >> work in a limited range of locales and/or enco

Re: [HACKERS] VARIANT / ANYTYPE datatype

2011-05-10 Thread Bruce Momjian
Darren Duncan wrote: > To follow-up, an additional feature that would be useful and resembles union > types is the variant where you could declare a union type first and then > separately other types could declare they are a member of the union. I'm > talking about loosely what mixins or type-r

Re: [HACKERS] VARIANT / ANYTYPE datatype

2011-05-10 Thread Alvaro Herrera
Excerpts from Bruce Momjian's message of mar may 10 16:21:36 -0400 2011: > Darren Duncan wrote: > > To follow-up, an additional feature that would be useful and resembles > > union > > types is the variant where you could declare a union type first and then > > separately other types could decla

Re: [HACKERS] VARIANT / ANYTYPE datatype

2011-05-10 Thread Darren Duncan
Alvaro Herrera wrote: Excerpts from Bruce Momjian's message of mar may 10 16:21:36 -0400 2011: Darren Duncan wrote: To follow-up, an additional feature that would be useful and resembles union types is the variant where you could declare a union type first and then separately other types could

Re: [HACKERS] Why not install pgstattuple by default?

2011-05-10 Thread Bruce Momjian
Tom Lane wrote: > Christopher Browne writes: > > But people are evidently still setting packaging policies based on how > > things were back in 7.3, even though that perhaps isn't necessary > > anymore. > > FWIW, once you get past the client versus server distinction, I think > most subpackaging

Re: [HACKERS] Backpatching of "Teach the regular expression functions to do case-insensitive matching"

2011-05-10 Thread Peter Eisentraut
On tis, 2011-05-10 at 15:17 -0400, Tom Lane wrote: > Well, that would be great, but the "someone" is not going to be me; > I don't do Windows. Yeah, me neither. At least not for this release. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscrip

Re: [HACKERS] Backpatching of "Teach the regular expression functions to do case-insensitive matching"

2011-05-10 Thread Peter Eisentraut
On tis, 2011-05-10 at 15:48 -0400, Robert Haas wrote: > On Tue, May 10, 2011 at 3:09 PM, Peter Eisentraut wrote: > > Well, the result of "people don't always run them" is the rest of > > src/test/. How much of that stuff even works anymore? > > I don't know. But I'm not sure I see your point.

Re: [HACKERS] Why not install pgstattuple by default?

2011-05-10 Thread Tom Lane
Bruce Momjian writes: > Tom Lane wrote: >> here are the sizes of the built RPMs from my last build for Fedora: >> >> -rw-r--r--. 1 tgl tgl 3839458 Apr 18 10:50 >> postgresql-9.0.4-1.fc13.x86_64.rpm >> -rw-r--r--. 1 tgl tgl 490788 Apr 18 10:50 >> postgresql-contrib-9.0.4-1.fc13.x86_64.rpm >>

Re: [HACKERS] VARIANT / ANYTYPE datatype

2011-05-10 Thread Simon Riggs
On Wed, May 4, 2011 at 5:58 PM, Alvaro Herrera wrote: > A customer came to us with this request: a way to store "any" data in a > column.  We've gone back and forth trying to determine reasonable > implementation restrictions, safety and useful semantics for them. > I note that this has been requ

[HACKERS] Infinity bsearch crash on Windows

2011-05-10 Thread Greg Smith
A 9.1Beta1 test report from Richard Broersma (and confirmed on another system by Mark Watson) showed up pgsql-testers this week at http://archives.postgresql.org/pgsql-testers/2011-05/msg0.php with the following test crashing his Windows server every time: SELECT 'INFINITY'::TIMESTAMP; Th

Re: [HACKERS] Backpatching of "Teach the regular expression functions to do case-insensitive matching"

2011-05-10 Thread Tom Lane
Peter Eisentraut writes: > I'm all for more test suites, but we should make them as widely > accessible and accessed as possible so that they get maintained. Yeah. My preference would really be to push something like collate.linux.utf8 into the standard regression tests, but we'd first have to g

Re: [HACKERS] VARIANT / ANYTYPE datatype

2011-05-10 Thread Eric McKeeth
On Tue, May 10, 2011 at 3:57 PM, Simon Riggs wrote: > On Wed, May 4, 2011 at 5:58 PM, Alvaro Herrera > wrote: > > > A customer came to us with this request: a way to store "any" data in a > > column. We've gone back and forth trying to determine reasonable > > implementation restrictions, safet

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Simon Riggs
On Tue, May 10, 2011 at 8:35 PM, Kevin Grittner wrote: > Simon Riggs wrote: >> Kevin Grittner wrote: >> ... but I share Simon's desire to see some proof before anything gets committed. >>> >>> And we agree there.  In fact, I can't think of anyone in the >>> community who doesn't want t

Re: [HACKERS] Feature proposal: distinguish each PostgreSQL instance in the event log

2011-05-10 Thread MauMau
From: "Tom Lane" "MauMau" writes: I've encountered one problem on Windows. I need to support running all of my products on one host simultaneously. Plus, I need to log messages in syslog/event log. On Linux, I can distinguish the messages of one product and those of other products by setting

Re: [HACKERS] Infinity bsearch crash on Windows

2011-05-10 Thread Peter Geoghegan
On 10 May 2011 23:02, Greg Smith wrote: > Why crash there only on Windows?  Was the problem actually introduced above > this part of the code?  These are all questions I have no answer for. I don't find it at all surprising that there's a memory corruption bug that only manifests itself on Window

Re: [HACKERS] Feature proposal: distinguish each PostgreSQL instance in the event log

2011-05-10 Thread Tom Lane
"MauMau" writes: >> "MauMau" writes: >>> I've encountered one problem on Windows. I need to support running all of >>> my >>> products on one host simultaneously. Plus, I need to log messages in >>> syslog/event log. On Linux, I can distinguish the messages of one product >>> and those of other

Re: [HACKERS] Infinity bsearch crash on Windows

2011-05-10 Thread Tom Lane
Greg Smith writes: > A 9.1Beta1 test report from Richard Broersma (and confirmed on another > system by Mark Watson) showed up pgsql-testers this week at > http://archives.postgresql.org/pgsql-testers/2011-05/msg0.php with > the following test crashing his Windows server every time: > SELE

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Kevin Grittner
Simon Riggs wrote: > Normally, others come forward with the why? when? questions and it > feels like there's a bit of groupthink going on here. This looks > to me like its being approached like it was a feature, but it > looks to me like a possible optimisation, so suggest we treat it > that way

Re: [HACKERS] Re: [BUGS] BUG #5957: createdb with description and md5 auth forces to provide password twice

2011-05-10 Thread Bruce Momjian
Bruce Momjian wrote: > Tom Lane wrote: > > Bruce Momjian writes: > > > Prior to PG 8.2, this was necessary to put the comment on the database, > > > but now that we have the shared comment/description table > > > pg_shdescription, this is not necessary. > > > > > Do we need createdb to be able to

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Greg Stark
On Wed, May 11, 2011 at 12:14 AM, Kevin Grittner wrote: > The problem is that there are regular and fairly frequent complaints > on the list about queries which run slower than people expect > To be fair about 3/4 of them were actually complaining about the lack of some global materialized cache

Re: [HACKERS] postgresql.conf error checking strategy

2011-05-10 Thread Bruce Momjian
Robert Haas wrote: > On Mon, May 9, 2011 at 11:10 AM, Tom Lane wrote: > > Robert Haas writes: > >> On Sun, May 8, 2011 at 1:04 AM, Tom Lane wrote: > >>> Yes, definitely. ?Perhaps summarize as "rethink how we handle partially > >>> correct postgresql.conf files". ?Or maybe Robert sees it as "reth

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Bruce Momjian
Robert Haas wrote: > >> Any thoughts welcome. ?Incidentally, if anyone else feels like working > >> on this, feel free to let me know and I'm happy to step away, from all > >> of it or from whatever part someone else wants to tackle. ?I'm mostly > >> working on this because it's something that I th

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Bruce Momjian
Greg Stark wrote: > On a separate note though, Simon, I don't know what you mean by "we > normally start with a problem". It's an free software project and > people are free to work on whatever interests them whether that's > because it solves a problem they have, helps a client who's paying > them

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Greg Stark
On Wed, May 11, 2011 at 1:47 AM, Bruce Momjian wrote: > Isn't speeding up COUNT(*) a sufficient case because it will not have to > touch the heap in many cases? Putting aside the politics questions, count(*) is an interesting case -- it exposes some of the unanswered questions about index-only sc

Re: [HACKERS] Why not install pgstattuple by default?

2011-05-10 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > > Tom Lane wrote: > >> here are the sizes of the built RPMs from my last build for Fedora: > >> > >> -rw-r--r--. 1 tgl tgl 3839458 Apr 18 10:50 > >> postgresql-9.0.4-1.fc13.x86_64.rpm > >> -rw-r--r--. 1 tgl tgl 490788 Apr 18 10:50 > >> postgresql-con

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Bruce Momjian
Greg Stark wrote: > On Wed, May 11, 2011 at 1:47 AM, Bruce Momjian wrote: > > Isn't speeding up COUNT(*) a sufficient case because it will not have to > > touch the heap in many cases? > > Putting aside the politics questions, count(*) is an interesting case > -- it exposes some of the unanswered

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Bruce Momjian
Robert Haas wrote: > So, what do we need in order to find our way to index-only scans? > > 1. The visibility map needs to be crash-safe. The basic idea of > index-only scans is that, instead of checking the heap to find out > whether each tuple is visible, we first check the visibility map. If >

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Bruce Momjian
Greg Stark wrote: > On Wed, May 11, 2011 at 1:47 AM, Bruce Momjian wrote: > > Isn't speeding up COUNT(*) a sufficient case because it will not have to > > touch the heap in many cases? > > Putting aside the politics questions, count(*) is an interesting case > -- it exposes some of the unanswered

[HACKERS] PGC_S_DEFAULT is inadequate

2011-05-10 Thread Tom Lane
I believe I've sussed the reason for the recent reports of Windows builds crashing when asked to process 'infinity'::timestamp. It's a bit tedious, so bear with me: 1. The immediate cause is that datebsearch() is being called with a NULL pointer and zero count, ie, the powerup default values of t

  1   2   >