Re: [HACKERS] Proposal: Incremental Backup

2014-08-07 Thread Benedikt Grundmann
On Thu, Aug 7, 2014 at 6:29 PM, Gabriele Bartolini < gabriele.bartol...@2ndquadrant.it> wrote: > Hi Marco, > > > With the current full backup procedure they are backed up, so I think > > that having them backed up with a rsync-like algorithm is what an user > > would expect for an incremental back

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-08-07 Thread Ashutosh Bapat
On Fri, Aug 8, 2014 at 10:48 AM, Stephen Frost wrote: > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > I looked into the issue reported in bug #11109. The problem appears to > be > > that jsonb's on-disk format is designed in such a way that the leading > > portion of any JSON array or object will b

Re: [HACKERS] Introducing coarse grain parallelism by postgres_fdw.

2014-08-07 Thread Ashutosh Bapat
On Fri, Aug 8, 2014 at 8:54 AM, Kyotaro HORIGUCHI < horiguchi.kyot...@lab.ntt.co.jp> wrote: > Hi, thank you for the comment. > > > Hi Kyotaro, > > I looked at the patches and felt that the approach taken here is too > > intrusive, considering that the feature is only for foreign scans. > > I agree

Re: [HACKERS] postgresql.auto.conf and reload

2014-08-07 Thread Fujii Masao
On Fri, Aug 8, 2014 at 1:19 PM, Amit Kapila wrote: > On Thu, Aug 7, 2014 at 12:36 PM, Fujii Masao wrote: >> On Thu, Aug 7, 2014 at 12:36 PM, Amit Kapila >> wrote: >> > On Wed, Aug 6, 2014 at 1:32 PM, Fujii Masao >> > wrote: > >> >> What about picking up only data_directory at the first pass? >>

Re: [HACKERS] Specifying the unit in storage parameter

2014-08-07 Thread Fujii Masao
On Fri, Aug 8, 2014 at 2:12 PM, Alvaro Herrera wrote: > Fujii Masao wrote: >> On Fri, Aug 8, 2014 at 12:56 PM, Alvaro Herrera >> wrote: > >> > Hm, what's with the parse_int signature change and the hintmsg thing? >> > Is it just me or the patch is incomplete? >> >> Sorry, probably I failed to see

Re: [HACKERS] Enhancing pgbench parameter checking

2014-08-07 Thread Tatsuo Ishii
Fabien, > Hello Tatsuo-san, > >> Thanks for the review. I have registered it to Aug Commit fest. >> https://commitfest.postgresql.org/action/patch_view?id=1532 >> >>> I'm not sure of the variable name "is_non_init_parameter_set". I would >>> suggest "benchmarking_option_set"? >> >> Ok, I will rep

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-08-07 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > I looked into the issue reported in bug #11109. The problem appears to be > that jsonb's on-disk format is designed in such a way that the leading > portion of any JSON array or object will be fairly incompressible, because > it consists mostly of a strictl

Re: [HACKERS] Specifying the unit in storage parameter

2014-08-07 Thread Alvaro Herrera
Fujii Masao wrote: > On Fri, Aug 8, 2014 at 12:56 PM, Alvaro Herrera > wrote: > > Hm, what's with the parse_int signature change and the hintmsg thing? > > Is it just me or the patch is incomplete? > > Sorry, probably I failed to see your point. You mean that the signature > of parse_int needs t

Re: [HACKERS] Hokey wrong versions of libpq in apt.postgresql.org

2014-08-07 Thread Stephen Frost
JD, * Joshua D. Drake (j...@commandprompt.com) wrote: > But this is just plain wrong. I don't care that the FAQ (on the > wiki) says we are doing it wrong for good reasons. When I (or anyone > else) pulls postgresql-$version-dev, I want the libpq for my > version. I do not want 9.3. No, it isn't

Re: [HACKERS] Specifying the unit in storage parameter

2014-08-07 Thread Fujii Masao
On Fri, Aug 8, 2014 at 12:56 PM, Alvaro Herrera wrote: > Fujii Masao wrote: >> Hi, >> >> We can specify the unit when setting autovacuum_vacuum_cost_delay >> GUC as follows. >> >> ALTER SYSTEM SET autovacuum_vacuum_cost_delay TO '80ms'; >> >> OTOH we cannot specify the unit when setting autova

Re: [HACKERS] postgresql.auto.conf and reload

2014-08-07 Thread Amit Kapila
On Thu, Aug 7, 2014 at 12:36 PM, Fujii Masao wrote: > On Thu, Aug 7, 2014 at 12:36 PM, Amit Kapila wrote: > > On Wed, Aug 6, 2014 at 1:32 PM, Fujii Masao wrote: > >> What about picking up only data_directory at the first pass? > > > > I think that will workout and for that I think we might need

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-08-07 Thread Larry White
Apologies if this is a ridiculous suggestion, but I believe that swapping out the compression algorithm (for Snappy, for example) has been discussed in the past. I wonder if that algorithm is sufficiently different that it would produce a better result, and if that might not be preferable to some o

Re: [HACKERS] Specifying the unit in storage parameter

2014-08-07 Thread Alvaro Herrera
Fujii Masao wrote: > Hi, > > We can specify the unit when setting autovacuum_vacuum_cost_delay > GUC as follows. > > ALTER SYSTEM SET autovacuum_vacuum_cost_delay TO '80ms'; > > OTOH we cannot specify the unit when setting autovacuum_vacuum_cost_delay > as storage parameter as follows. > >

Re: [HACKERS] Use unique index for longer pathkeys.

2014-08-07 Thread Kyotaro HORIGUCHI
Hello, > > Although, yes, you're right, irrespective of the "common > > something", and even if the dropped index was i_t1_pkey_2, which > > is on t1(a, b), the result tuples are sorted as expected only by > > the pathkey (t.a = t1.a, t1.b). It is because both t and t1 are > > still unique so the

[HACKERS] Specifying the unit in storage parameter

2014-08-07 Thread Fujii Masao
Hi, We can specify the unit when setting autovacuum_vacuum_cost_delay GUC as follows. ALTER SYSTEM SET autovacuum_vacuum_cost_delay TO '80ms'; OTOH we cannot specify the unit when setting autovacuum_vacuum_cost_delay as storage parameter as follows. CREATE TABLE test (col1 int) WITH (au

Re: [HACKERS] Introducing coarse grain parallelism by postgres_fdw.

2014-08-07 Thread Kyotaro HORIGUCHI
Hi, thank you for the comment. > Hi Kyotaro, > I looked at the patches and felt that the approach taken here is too > intrusive, considering that the feature is only for foreign scans. I agree to you premising that it's only for foreign scans but I regard it as an example of parallel execution pl

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Craig Ringer
On 08/08/2014 09:51 AM, Tom Lane wrote: >> AFAIK we don't _have_ a fancy negotiation system in the protocol, with >> back-and-forth exchanges of capabilities information. > > Maybe it's time to invent that. It would be positively foolish to > create any such behavior without a protocol version b

[HACKERS] jsonb format is pessimal for toast compression

2014-08-07 Thread Tom Lane
I looked into the issue reported in bug #11109. The problem appears to be that jsonb's on-disk format is designed in such a way that the leading portion of any JSON array or object will be fairly incompressible, because it consists mostly of a strictly-increasing series of integer offsets. This in

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Michael Paquier
On Fri, Aug 8, 2014 at 11:58 AM, Fujii Masao wrote: > On Fri, Aug 8, 2014 at 9:50 AM, Craig Ringer wrote: > ISTM that the proper solution to that problem is the introduction of > new synchronous replication mode which makes the transaction wait for > its WAL to be replayed by the standby. If this

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Craig Ringer
On 08/08/2014 10:58 AM, Fujii Masao wrote: > ISTM that the proper solution to that problem is the introduction of > new synchronous replication mode which makes the transaction wait for > its WAL to be replayed by the standby. If this mode is used, a client > doesn't need to track the LSN of each t

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Fujii Masao
On Fri, Aug 8, 2014 at 9:50 AM, Craig Ringer wrote: > On 08/08/2014 03:54 AM, Tom Lane wrote: >> Craig Ringer writes: >>> Hi all >>> To support transparent client-side failover in BDR, it's necessary to >>> know what the LSN of a node was at the time a transaction committed and >>> keep track of

Re: [HACKERS] Minmax indexes

2014-08-07 Thread Josh Berkus
On 08/07/2014 05:52 PM, Michael Paquier wrote: > On Fri, Aug 8, 2014 at 9:47 AM, Josh Berkus wrote: >> On 08/07/2014 08:38 AM, Oleg Bartunov wrote: >>> +1 for BRIN ! >>> >>> On Thu, Aug 7, 2014 at 6:16 PM, Simon Riggs wrote: On 7 August 2014 14:53, Robert Haas wrote: A better descripti

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Craig Ringer
On 08/08/2014 09:02 AM, Tom Lane wrote: > Craig Ringer writes: >> On 08/08/2014 03:54 AM, Tom Lane wrote: >>> FWIW, I think it's a seriously bad idea to expose LSNs in the protocol >>> at all. What happens five years from now when we switch to some other >>> implementation that doesn't have LSNs

Re: [HACKERS] Fixed redundant i18n strings in json

2014-08-07 Thread David Johnston
On Thu, Aug 7, 2014 at 5:53 PM, Tom Lane wrote: > David G Johnston writes: > > Tom Lane-2 wrote > >> Surely that was meant to read "invalid number OF arguments". The > errhint > >> is only charitably described as English, as well. I'd suggest something > >> like "Arguments of json_build_object

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Craig Ringer
On 08/07/2014 11:42 PM, Robert Haas wrote: > I doubt whether it makes sense to do this without a broader > understanding of how the client-side failover mechanism would work. > If we're going to add something like this, it should include libpq > support for actually doing something useful with it.

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Tom Lane
Craig Ringer writes: > On 08/08/2014 03:54 AM, Tom Lane wrote: >> FWIW, I think it's a seriously bad idea to expose LSNs in the protocol >> at all. What happens five years from now when we switch to some other >> implementation that doesn't have LSNs? > Everyone who's relying on them already vi

Re: [HACKERS] Minmax indexes

2014-08-07 Thread Peter Geoghegan
On Thu, Aug 7, 2014 at 7:58 AM, Robert Haas wrote: > range index might get confused with range types; block range index > seems better. I like summary, but I'm fine with block range index or > block filter index, too. +1 -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hack

Re: [HACKERS] Fixed redundant i18n strings in json

2014-08-07 Thread Tom Lane
David G Johnston writes: > Tom Lane-2 wrote >> Surely that was meant to read "invalid number OF arguments". The errhint >> is only charitably described as English, as well. I'd suggest something >> like "Arguments of json_build_object() must be pairs of keys and values." >> --- but maybe someone

Re: [HACKERS] Minmax indexes

2014-08-07 Thread Michael Paquier
On Fri, Aug 8, 2014 at 9:47 AM, Josh Berkus wrote: > On 08/07/2014 08:38 AM, Oleg Bartunov wrote: >> +1 for BRIN ! >> >> On Thu, Aug 7, 2014 at 6:16 PM, Simon Riggs wrote: >>> On 7 August 2014 14:53, Robert Haas wrote: >>> A better description would be "block range index" since we are >>> indexi

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Craig Ringer
On 08/08/2014 03:54 AM, Tom Lane wrote: > Craig Ringer writes: >> Hi all >> To support transparent client-side failover in BDR, it's necessary to >> know what the LSN of a node was at the time a transaction committed and >> keep track of that in the client/proxy. > >> I'm thinking about adding a

Re: [HACKERS] Minmax indexes

2014-08-07 Thread Josh Berkus
On 08/07/2014 08:38 AM, Oleg Bartunov wrote: > +1 for BRIN ! > > On Thu, Aug 7, 2014 at 6:16 PM, Simon Riggs wrote: >> On 7 August 2014 14:53, Robert Haas wrote: >> A better description would be "block range index" since we are >> indexing a range of blocks (not just one block). Perhaps a better

Re: [HACKERS] Quick doc fix

2014-08-07 Thread Tom Lane
Guillaume Lelarge writes: > Still translating the 9.4 manual, and found another typo. Patch attached. Applied, thanks! regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.or

Re: [HACKERS] PostrgeSQL vs oracle doing 1 million sqrts am I doing it wrong?

2014-08-07 Thread Josh Berkus
On 08/07/2014 04:48 PM, Tom Lane wrote: > plpgsql is not efficient at all about coercions performed as a side > effect of assignments; if memory serves, it always handles them by > converting to text and back. So basically the added cost here came > from float8out() and float4in(). There has been

Re: [HACKERS] PostrgeSQL vs oracle doing 1 million sqrts am I doing it wrong?

2014-08-07 Thread Tom Lane
James Cloos writes: > "ST" == Shaun Thomas writes: > ST> That said, the documentation here says FLOAT4 is an alias for REAL, > ST> so it's somewhat nonintuitive for FLOAT4 to be so much slower than > ST> FLOAT8, which is an alias for DOUBLE PRECISION. > There are some versions of glibc where doi

[HACKERS] Hokey wrong versions of libpq in apt.postgresql.org

2014-08-07 Thread Joshua D. Drake
Hello, I know this has been brought up before: http://www.postgresql.org/message-id/20140724080902.ga28...@msg.df7cb.de But this is just plain wrong. I don't care that the FAQ (on the wiki) says we are doing it wrong for good reasons. When I (or anyone else) pulls postgresql-$version-dev, I

Re: [HACKERS] Append to a GUC parameter ?

2014-08-07 Thread Jerry Sievers
Alvaro Herrera writes: > Fabrízio de Royes Mello wrote: > >> On Tue, Aug 5, 2014 at 10:55 PM, Tom Lane wrote: >> >> > Josh Berkus writes: >> > > BTW, while there's unlikely to be a good reason to put search_path in >> > > pg.conf with appends, there are a LOT of reasons to want to be able to >

Re: [HACKERS] A worst case for qsort

2014-08-07 Thread Peter Geoghegan
On Thu, Aug 7, 2014 at 2:34 PM, Rod Taylor wrote: > This one is frequently sorted as batch operations against the files are > performed in alphabetical order to reduce conflict issues that a random > ordering may cause between jobs. Sure. There are cases out there. But, again, I have a hard time

Re: [HACKERS] A worst case for qsort

2014-08-07 Thread Rod Taylor
Sigh. Found another example. A table with 15 million entries and a unique key on filesystem location for things users created via a web interface. Entries all begin with /usr/home/ ... This one is frequently sorted as batch operations against the files are performed in alphabetical order to redu

Re: [HACKERS] A worst case for qsort

2014-08-07 Thread Peter Geoghegan
On Thu, Aug 7, 2014 at 2:23 PM, Rod Taylor wrote: > While I'm sure it's not common, I've seen a couple of ten-million tuple > tables having a URL column as primary key where 98% of the entries begin > with 'http://www.' > > So, that exact scenario is out there. Sure, that scenario is relatively c

Re: [HACKERS] Fixed redundant i18n strings in json

2014-08-07 Thread David G Johnston
Tom Lane-2 wrote > Robert Haas < > robertmhaas@ > > writes: >> On Sat, Aug 2, 2014 at 9:15 AM, Daniele Varrazzo >> < > daniele.varrazzo@ > > wrote: >>> I'd definitely replace /arg/argument/. Furthermore I'd avoid the form >>> "argument 1: something is wrong": the string is likely to end up in >

Re: [HACKERS] A worst case for qsort

2014-08-07 Thread Rod Taylor
On Thu, Aug 7, 2014 at 3:06 PM, Peter Geoghegan wrote: > I think that pre-sorted, all-unique text datums, that have all > differences beyond the first 8 bytes, that the user happens to > actually want to sort are fairly rare. While I'm sure it's not common, I've seen a couple of ten-million tup

Re: [HACKERS] A worst case for qsort

2014-08-07 Thread Peter Geoghegan
On Thu, Aug 7, 2014 at 12:06 PM, Peter Geoghegan wrote: > I think that pre-sorted, all-unique text datums, that have all > differences beyond the first 8 bytes, that the user happens to > actually want to sort are fairly rare. Actually, you could use that case to justify not doing strxfrm() trans

[HACKERS] Quick doc fix

2014-08-07 Thread Guillaume Lelarge
Hi, Still translating the 9.4 manual, and found another typo. Patch attached. Thanks. -- Guillaume. http://blog.guillaume.lelarge.info http://www.dalibo.com diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml index d3fcb82..cf174f0 100644 --- a/doc/src/sgml/maintenan

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-08-07 Thread Peter Geoghegan
On Thu, Aug 7, 2014 at 12:17 PM, Robert Haas wrote: > Gah. Hit send to soon. Also, as much as I'd prefer to avoid > relitigating the absolutely stupid debate about how to expand the > buffers, this is no good: > > + tss->buflen1 = Max(len1 + 1, tss->buflen1 * 2); > > If the first e

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Tom Lane
Craig Ringer writes: > Hi all > To support transparent client-side failover in BDR, it's necessary to > know what the LSN of a node was at the time a transaction committed and > keep track of that in the client/proxy. > I'm thinking about adding a new message type in the protocol that gets > sent

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-08-07 Thread Peter Geoghegan
On Thu, Aug 7, 2014 at 12:15 PM, Robert Haas wrote: > In my original patch, I wrote NUL, as in the NUL character. You've > changed it to NULL, but the original was correct. NULL is a pointer > value that is not relevant here; the character with value 0 is NUL. "NULL-terminated string" seems lik

Re: [HACKERS] psql: show only failed queries

2014-08-07 Thread Pavel Stehule
2014-08-07 7:10 GMT+02:00 Fujii Masao : > On Thu, Aug 7, 2014 at 6:26 AM, Pavel Stehule > wrote: > > Hello > > > > updated version patch in attachment > > Thanks! But ISTM you forgot to attached the patch. > grr .. I am sorry > > >> +/* all psql known variables are included in list by defa

Re: [HACKERS] Fixed redundant i18n strings in json

2014-08-07 Thread Tom Lane
Robert Haas writes: > On Sat, Aug 2, 2014 at 9:15 AM, Daniele Varrazzo > wrote: >> I'd definitely replace /arg/argument/. Furthermore I'd avoid the form >> "argument 1: something is wrong": the string is likely to end up in >> sentences with other colons so I'd rather use "something is wrong at >

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-08-07 Thread Robert Haas
On Thu, Aug 7, 2014 at 3:15 PM, Robert Haas wrote: > On Wed, Aug 6, 2014 at 7:18 PM, Peter Geoghegan wrote: >> On Wed, Aug 6, 2014 at 1:11 PM, Robert Haas wrote: >>> I've committed the patch I posted yesterday. I did not see a good >>> reason to meld that together in a single commit with the fi

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-08-07 Thread Robert Haas
On Wed, Aug 6, 2014 at 7:18 PM, Peter Geoghegan wrote: > On Wed, Aug 6, 2014 at 1:11 PM, Robert Haas wrote: >> I've committed the patch I posted yesterday. I did not see a good >> reason to meld that together in a single commit with the first of the >> patches you posted; I'll leave you to revis

Re: [HACKERS] A worst case for qsort

2014-08-07 Thread Peter Geoghegan
On Thu, Aug 7, 2014 at 11:33 AM, Robert Haas wrote: > I think that's actually not a very unrealistic case at all. In > general, I think that if a particular data distribution is a > reasonable scenario, that data distribution plus "it's already sorted" > is also reasonable. Data gets presorted i

Re: [HACKERS] Pg_upgrade and toast tables bug discovered

2014-08-07 Thread Bruce Momjian
On Tue, Aug 5, 2014 at 07:31:21PM -0400, Bruce Momjian wrote: > On Thu, Jul 10, 2014 at 06:38:26PM -0400, Bruce Momjian wrote: > > On Thu, Jul 10, 2014 at 06:17:14PM -0400, Bruce Momjian wrote: > > > Well, we are going to need to call internal C functions, often bypassing > > > their typical call

Re: [HACKERS] A worst case for qsort

2014-08-07 Thread Robert Haas
On Thu, Aug 7, 2014 at 1:16 PM, Peter Geoghegan wrote: > On Thu, Aug 7, 2014 at 8:07 AM, Robert Haas wrote: >> So here. You may not agree that the mitigation strategies for which >> others are asking for are worthwhile, but you can't expect everyone >> else to agree with your assessment of which

Re: [HACKERS] Minmax indexes

2014-08-07 Thread Petr Jelinek
On 07/08/14 16:16, Simon Riggs wrote: A better description would be "block range index" since we are indexing a range of blocks (not just one block). Perhaps a better one would be simply "range index", which we could abbreviate to RIN or BRIN. +1 for block range index (BRIN) -- Petr Jelinek

Re: [HACKERS] Minmax indexes

2014-08-07 Thread Nicolas Barbier
2014-08-07 Oleg Bartunov : > +1 for BRIN ! +1, rolls off the tongue smoothly and captures the essence :-). Nicolas -- A. Because it breaks the logical sequence of discussion. Q. Why is top posting bad? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] Proposal: Incremental Backup

2014-08-07 Thread Gabriele Bartolini
Hi Marco, > With the current full backup procedure they are backed up, so I think > that having them backed up with a rsync-like algorithm is what an user > would expect for an incremental backup. Exactly. I think a simple, flexible and robust method for file based incremental backup is all we ne

Re: [HACKERS] A worst case for qsort

2014-08-07 Thread Peter Geoghegan
On Thu, Aug 7, 2014 at 8:07 AM, Robert Haas wrote: > So here. You may not agree that the mitigation strategies for which > others are asking for are worthwhile, but you can't expect everyone > else to agree with your assessment of which cases are likely to occur > in practice. The case of a coho

Re: [HACKERS] A worst case for qsort

2014-08-07 Thread John Cochran
On Thu, Aug 7, 2014 at 11:07 AM, Robert Haas wrote: > On Tue, Aug 5, 2014 at 8:15 PM, Peter Geoghegan wrote: > > "The adversarial method works for almost any polymorphic program > > recognizable as quicksort. The subject quicksort may copy values at > > will, or work with lists rather than array

Re: [HACKERS] replication commands and log_statements

2014-08-07 Thread Abhijit Menon-Sen
At 2014-08-07 23:22:43 +0900, masao.fu...@gmail.com wrote: > > That is, we log replication commands only when log_statement is set to > all. Neither new parameter is introduced nor log_statement is > redefined as a list. That sounds good to me. -- Abhijit -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Proposal: Incremental Backup

2014-08-07 Thread Marco Nenciarini
Il 07/08/14 17:25, Bruce Momjian ha scritto: > On Thu, Aug 7, 2014 at 08:35:53PM +0900, Michael Paquier wrote: >> On Thu, Aug 7, 2014 at 8:11 PM, Fujii Masao wrote: >>> There are some data which don't have LSN, for example, postgresql.conf. >>> When such data has been modified since last backup,

Re: [HACKERS] Proposal: Incremental Backup

2014-08-07 Thread Marco Nenciarini
Il 07/08/14 17:29, Bruce Momjian ha scritto: > I am a little worried that many users will not realize this until they > try it and are disappointed, e.g. "Why is PG writing to my static data > so often?" --- then we get beaten up about our hint bits and freezing > behavior. :-( > > I am just tryi

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Robert Haas
On Wed, Aug 6, 2014 at 9:15 PM, Craig Ringer wrote: > To support transparent client-side failover in BDR, it's necessary to > know what the LSN of a node was at the time a transaction committed and > keep track of that in the client/proxy. I doubt whether it makes sense to do this without a broad

Re: [HACKERS] Minmax indexes

2014-08-07 Thread Oleg Bartunov
+1 for BRIN ! On Thu, Aug 7, 2014 at 6:16 PM, Simon Riggs wrote: > On 7 August 2014 14:53, Robert Haas wrote: >> On Wed, Aug 6, 2014 at 4:06 PM, Nicolas Barbier >> wrote: >>> 2014-08-06 Claudio Freire : >>> So, I like blockfilter a lot. I change my vote to blockfilter ;) >>> >>> +1 for blo

Re: [HACKERS] Proposal: Incremental Backup

2014-08-07 Thread Bruce Momjian
On Thu, Aug 7, 2014 at 11:03:40AM +0100, Simon Riggs wrote: > Well, there is a huge difference between file-level and block-level backup. > > Designing, writing and verifying block-level backup to the point that > it is acceptable is a huge effort. (Plus, I don't think accumulating > block number

Re: [HACKERS] Proposal: Incremental Backup

2014-08-07 Thread Bruce Momjian
On Thu, Aug 7, 2014 at 08:35:53PM +0900, Michael Paquier wrote: > On Thu, Aug 7, 2014 at 8:11 PM, Fujii Masao wrote: > > There are some data which don't have LSN, for example, postgresql.conf. > > When such data has been modified since last backup, they also need to > > be included in incremental

Re: [HACKERS] Enhancing pgbench parameter checking

2014-08-07 Thread Fabien COELHO
Hello Tatsuo-san, Thanks for the review. I have registered it to Aug Commit fest. https://commitfest.postgresql.org/action/patch_view?id=1532 I'm not sure of the variable name "is_non_init_parameter_set". I would suggest "benchmarking_option_set"? Ok, I will replace the variable name as you

Re: [HACKERS] A worst case for qsort

2014-08-07 Thread Robert Haas
On Tue, Aug 5, 2014 at 8:15 PM, Peter Geoghegan wrote: > "The adversarial method works for almost any polymorphic program > recognizable as quicksort. The subject quicksort may copy values at > will, or work with lists rather than arrays. It may even pick the > pivot at random. The quicksort will

Re: [HACKERS] Minmax indexes

2014-08-07 Thread Robert Haas
On Thu, Aug 7, 2014 at 10:16 AM, Simon Riggs wrote: > On 7 August 2014 14:53, Robert Haas wrote: >> On Wed, Aug 6, 2014 at 4:06 PM, Nicolas Barbier >> wrote: >>> 2014-08-06 Claudio Freire : >>> So, I like blockfilter a lot. I change my vote to blockfilter ;) >>> >>> +1 for blockfilter, beca

Re: [HACKERS] Minmax indexes

2014-08-07 Thread Alvaro Herrera
Simon Riggs wrote: > On 7 August 2014 14:53, Robert Haas wrote: > > On Wed, Aug 6, 2014 at 4:06 PM, Nicolas Barbier > > wrote: > >> 2014-08-06 Claudio Freire : > >> > >>> So, I like blockfilter a lot. I change my vote to blockfilter ;) > >> > >> +1 for blockfilter, because it stresses the fact th

Re: [HACKERS] PostrgeSQL vs oracle doing 1 million sqrts am I doing it wrong?

2014-08-07 Thread Merlin Moncure
On Thu, Aug 7, 2014 at 5:12 AM, Craig Ringer wrote: > New Intel hardware supports IEEE 754:2008 decimal floating point in > hardware, and I'm quite interested in implementing DECFLOAT(n) for > PostgreSQL to take advantage of that. +1 merlin -- Sent via pgsql-hackers mailing list (pgsql-hacker

Re: [HACKERS] replication commands and log_statements

2014-08-07 Thread Fujii Masao
On Wed, Jul 2, 2014 at 4:26 AM, Abhijit Menon-Sen wrote: > Hi. > > Do we have any consensus about what to do with these two patches? > > 1. Introduce a "log_replication_command" setting. > 2. Change log_statement to be a list of tokens. > > If I understand correctly, there weren't any strong objec

Re: [HACKERS] delta relations in AFTER triggers

2014-08-07 Thread Kevin Grittner
Thanks for your review and comments, Amit! Amit Khandekar wrote: > On 21 June 2014 23:36, Kevin Grittner wrote: >> Kevin Grittner wrote: >> I didn't change the tuplestores to TID because it seemed to me that >> it would preclude using transition relations with FDW triggers, and >> it seemed bad

Re: [HACKERS] Minmax indexes

2014-08-07 Thread Claudio Freire
On Thu, Aug 7, 2014 at 11:16 AM, Simon Riggs wrote: > On 7 August 2014 14:53, Robert Haas wrote: >> On Wed, Aug 6, 2014 at 4:06 PM, Nicolas Barbier >> wrote: >>> 2014-08-06 Claudio Freire : >>> So, I like blockfilter a lot. I change my vote to blockfilter ;) >>> >>> +1 for blockfilter, beca

Re: [HACKERS] Minmax indexes

2014-08-07 Thread Simon Riggs
On 7 August 2014 14:53, Robert Haas wrote: > On Wed, Aug 6, 2014 at 4:06 PM, Nicolas Barbier > wrote: >> 2014-08-06 Claudio Freire : >> >>> So, I like blockfilter a lot. I change my vote to blockfilter ;) >> >> +1 for blockfilter, because it stresses the fact that the "physical" >> arrangement of

Re: [HACKERS] Append to a GUC parameter ?

2014-08-07 Thread Alvaro Herrera
Fabrízio de Royes Mello wrote: > On Tue, Aug 5, 2014 at 10:55 PM, Tom Lane wrote: > > > Josh Berkus writes: > > > BTW, while there's unlikely to be a good reason to put search_path in > > > pg.conf with appends, there are a LOT of reasons to want to be able to > > > append to it during a session

Re: [HACKERS] Minmax indexes

2014-08-07 Thread Robert Haas
On Wed, Aug 6, 2014 at 4:06 PM, Nicolas Barbier wrote: > 2014-08-06 Claudio Freire : > >> So, I like blockfilter a lot. I change my vote to blockfilter ;) > > +1 for blockfilter, because it stresses the fact that the "physical" > arrangement of rows in blocks matters for this index. I don't like

Re: [HACKERS] Fixed redundant i18n strings in json

2014-08-07 Thread Robert Haas
On Wed, Aug 6, 2014 at 7:20 PM, Jeff Janes wrote: > On Wed, Aug 6, 2014 at 8:35 AM, Robert Haas wrote: >> >> On Tue, Aug 5, 2014 at 6:05 PM, Jeff Janes wrote: >> > I think you missed one of the regression tests, see attached >> >> Woops. Thanks, committed. > > Thanks. > > It needs to go into 9_

Re: [HACKERS] Partitioning performance: cache stringToNode() of pg_constraint.ccbin

2014-08-07 Thread Robert Haas
On Wed, Aug 6, 2014 at 9:35 PM, Bruce Momjian wrote: > On Sun, Jan 12, 2014 at 12:53:40PM -0500, Noah Misch wrote: >> On Sat, Jan 11, 2014 at 02:10:01PM -0500, Bruce Momjian wrote: >> > On Mon, Jun 3, 2013 at 03:07:27PM -0400, Noah Misch wrote: >> > > A colleague, Korry Douglas, observed a table

Re: [HACKERS] HEAD crashes with assertion and LWLOCK_STATS enabled

2014-08-07 Thread Fujii Masao
On Fri, May 23, 2014 at 9:38 PM, Robert Haas wrote: > On Tue, May 20, 2014 at 4:02 AM, Yuto HAYAMIZU wrote: >> The failing assertion is for prohibiting memory allocation in a critical >> section, which is introduced by commit 4a170ee9 on 2014-04-04. This issue is still in open item list for 9.4

Re: [HACKERS] Wraparound limits

2014-08-07 Thread Heikki Linnakangas
On 08/07/2014 01:34 PM, Teodor Sigaev wrote: Hi! I have a questions about setting transaction's wraparound limits. Function SetTransactionIdLimit() in access/transam/varsup.c: 1) xidWrapLimit = oldest_datfrozenxid + (MaxTransactionId >> 1); if (xidWrapLimit < FirstNormalTransactionI

Re: [HACKERS] pg_shmem_allocations view

2014-08-07 Thread Michael Paquier
On Thu, May 8, 2014 at 10:28 PM, Andres Freund wrote: > Well, we have to live with it for now :) I just had a look at the first patch and got some comments: 1) Instead of using an assertion here, wouldn't it be better to error out if name is NULL, and truncate the name if it is longer than SHMEM_I

Re: [HACKERS] delta relations in AFTER triggers

2014-08-07 Thread Amit Khandekar
On 21 June 2014 23:36, Kevin Grittner wrote: > Kevin Grittner wrote: > I didn't change the tuplestores to TID because it seemed to me that > it would preclude using transition relations with FDW triggers, and > it seemed bad not to support that. Does anyone see a way around > that, or feel that

Re: [HACKERS] Proposal: Incremental Backup

2014-08-07 Thread Michael Paquier
On Thu, Aug 7, 2014 at 8:11 PM, Fujii Masao wrote: > There are some data which don't have LSN, for example, postgresql.conf. > When such data has been modified since last backup, they also need to > be included in incremental backup? Probably yes. Definitely yes. That's as well the case of paths l

Re: [HACKERS] Proposal: Incremental Backup

2014-08-07 Thread Fujii Masao
On Thu, Aug 7, 2014 at 12:20 AM, Bruce Momjian wrote: > On Wed, Aug 6, 2014 at 06:48:55AM +0100, Simon Riggs wrote: >> On 6 August 2014 03:16, Bruce Momjian wrote: >> > On Wed, Aug 6, 2014 at 09:17:35AM +0900, Michael Paquier wrote: >> >> On Wed, Aug 6, 2014 at 9:04 AM, Simon Riggs wrote: >> >

Re: [HACKERS] posix_fadvise() and pg_receivexlog

2014-08-07 Thread Fujii Masao
On Thu, Aug 7, 2014 at 5:02 PM, Heikki Linnakangas wrote: > On 08/07/2014 10:10 AM, Mitsumasa KONDO wrote: >> >> 2014-08-07 13:47 GMT+09:00 Fujii Masao : >> >>> On Thu, Aug 7, 2014 at 3:59 AM, Heikki Linnakangas >>> wrote: On 08/06/2014 08:39 PM, Fujii Masao wrote: > > The WAL f

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Ants Aasma
On Thu, Aug 7, 2014 at 4:15 AM, Craig Ringer wrote: > I'm thinking about adding a new message type in the protocol that gets > sent immediately before CommandComplete, containing the LSN of the > commit. Clients would need to enable the sending of this message with a > GUC that they set when they

[HACKERS] [GSoC] kmedoids status report

2014-08-07 Thread Maxence Ahlouche
Hi! Here is a report of what has been discussed yesterday on IRC. The kmedoids module now seems to work correctly on basic datasets. I've also implemented its variants with different seeding methods: random initial medoids, and initial medoids distributed among the points (similar to kmeans++ [0]

[HACKERS] Wraparound limits

2014-08-07 Thread Teodor Sigaev
Hi! I have a questions about setting transaction's wraparound limits. Function SetTransactionIdLimit() in access/transam/varsup.c: 1) xidWrapLimit = oldest_datfrozenxid + (MaxTransactionId >> 1); if (xidWrapLimit < FirstNormalTransactionId) xidWrapLimit += FirstNormalTransactio

Re: [HACKERS] PostrgeSQL vs oracle doing 1 million sqrts am I doing it wrong?

2014-08-07 Thread Craig Ringer
On 08/05/2014 10:44 PM, Shaun Thomas wrote: > On 08/05/2014 12:56 AM, Mark Kirkwood wrote: > >> The moral of the story for this case is that mapping Oracle to Postgres >> datatypes can require some careful thought. Using 'native' types (like >> integer, float8 etc) will generally give vastly quick

Re: [HACKERS] Proposal: Incremental Backup

2014-08-07 Thread Simon Riggs
On 6 August 2014 17:27, Bruce Momjian wrote: > On Wed, Aug 6, 2014 at 01:15:32PM -0300, Claudio Freire wrote: >> On Wed, Aug 6, 2014 at 12:20 PM, Bruce Momjian wrote: >> > >> > Well, for file-level backups we have: >> > >> > 1) use file modtime (possibly inaccurate) >> > 2) use f

Re: [HACKERS] Enhancing pgbench parameter checking

2014-08-07 Thread Tatsuo Ishii
Fabien, > I have not tested, but the patch looks ok in principle. Thanks for the review. I have registered it to Aug Commit fest. https://commitfest.postgresql.org/action/patch_view?id=1532 > I'm not sure of the variable name "is_non_init_parameter_set". I would > suggest "benchmarking_option_s

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-08-07 Thread furuyao
> Okay, applied the patch. > > I heavily modified your patch based on the master that the refactoring > patch has been applied. Attached is that modified version. Could you > review that? Thank you for the patch. I did a review of the patch. No problem in the patch. Behavior after the true re

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-08-07 Thread Peter Geoghegan
On Wed, Aug 6, 2014 at 10:36 PM, Peter Geoghegan wrote: > This *almost* applies to patched Postgres if you pick a benchmark that > is very sympathetic to my patch. To my surprise, work_mem = '10MB' > (which results in an external tape sort) is sometimes snapping at the > heels of a work_mem = '5GB

Re: [HACKERS] posix_fadvise() and pg_receivexlog

2014-08-07 Thread Heikki Linnakangas
On 08/07/2014 10:10 AM, Mitsumasa KONDO wrote: 2014-08-07 13:47 GMT+09:00 Fujii Masao : On Thu, Aug 7, 2014 at 3:59 AM, Heikki Linnakangas wrote: On 08/06/2014 08:39 PM, Fujii Masao wrote: The WAL files that pg_receivexlog writes will not be re-read soon basically, so we can advise the OS to

Re: [HACKERS] A worst case for qsort

2014-08-07 Thread Fabien COELHO
Hello John, [...] In fact, the mentioned paper says this about the subject "Moreover, if worst-case performance is important, Quicksort is the wrong algorithm." I fully agree with this conclusion. -- Fabien -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chan

Re: [HACKERS] A worst case for qsort

2014-08-07 Thread Fabien COELHO
IMHO, while worst case performance is a very useful tool for analyzing algorithms (particularly their worst case time complexity), a worst case should be put in its practical context. For example, if we had reason to be concerned about *adversarial* inputs, I think that there is a good chance th

Re: [HACKERS] posix_fadvise() and pg_receivexlog

2014-08-07 Thread Mitsumasa KONDO
Hi, 2014-08-07 13:47 GMT+09:00 Fujii Masao : > On Thu, Aug 7, 2014 at 3:59 AM, Heikki Linnakangas > wrote: > > On 08/06/2014 08:39 PM, Fujii Masao wrote: > >> The WAL files that pg_receivexlog writes will not be re-read soon > >> basically, > >> so we can advise the OS to release any cached page

Re: [HACKERS] postgresql.auto.conf and reload

2014-08-07 Thread Fujii Masao
On Thu, Aug 7, 2014 at 12:28 PM, Amit Kapila wrote: > On Wed, Aug 6, 2014 at 11:39 AM, Fujii Masao wrote: >> >> On Tue, Aug 5, 2014 at 12:49 PM, Fujii Masao >> wrote: >> > On Mon, Aug 4, 2014 at 11:52 PM, Tom Lane wrote: >> >> Fujii Masao writes: >> >>> The patch chooses the last settings for

Re: [HACKERS] postgresql.auto.conf and reload

2014-08-07 Thread Fujii Masao
On Thu, Aug 7, 2014 at 12:36 PM, Amit Kapila wrote: > On Wed, Aug 6, 2014 at 1:32 PM, Fujii Masao wrote: >> On Tue, Jul 29, 2014 at 9:26 PM, Amit Kapila >> wrote: >> > >> > The reason is that during startup DataDir is not set by the time server >> > processes config file due to which we process