[HACKERS] Re: [COMMITTERS] pgsql: Add API functions to libpq to interrogate SSL related stuff.

2015-02-03 Thread Heikki Linnakangas
On 02/04/2015 02:23 AM, Tom Lane wrote: Heikki Linnakangas writes: Add API functions to libpq to interrogate SSL related stuff. This patch is one large brick shy of a load: it creates exported libpq functions but fails to ensure they always exist. That's why jacana is unhappy; though TBH I'm

Re: [HACKERS] Fetch zero result rows when executing a query?

2015-02-03 Thread Jim Nasby
On 2/3/15 5:26 AM, Shay Rojansky wrote: Sorry if this has been asked before, couldn't find any mention... I'm working on the Npgsql, the .NET driver for PostgreSQL, and am trying to find a way to execute a query but without fetching any rows. The Execute message has a maximum result-row count, b

Re: [HACKERS] Table description in the data file (Re: pg_rawdump)

2015-02-03 Thread Jim Nasby
On 2/3/15 1:43 AM, Stephen R. van den Berg wrote: In relation to the talk and discussions at FOSDEM regarding helping data recovery, I searched the archives for the old thread after I performed my last recovery; for reference: http://www.postgresql.org/message-id/20101019201223.ga15...@cuci.nl

Re: [HACKERS] Proposal : REINDEX xxx VERBOSE

2015-02-03 Thread Tom Lane
Jim Nasby writes: > On 2/3/15 5:08 PM, Tom Lane wrote: >> Jim Nasby writes: >>> VACUUM puts the options before the table name, so ISTM it'd be best to >>> keep that with REINDEX. Either REINDEX (options) {INDEX | ...} or >>> REINDEX {INDEX | ...} (options). >> Well, I really really don't like th

Re: [HACKERS] Proposal : REINDEX xxx VERBOSE

2015-02-03 Thread Jim Nasby
On 2/3/15 5:08 PM, Tom Lane wrote: Jim Nasby writes: On 2/3/15 9:20 AM, Tom Lane wrote: Well, the object type is not an optional part of the command. It's *necessary*. I was thinking more like REINDEX { INDEX | TABLE | etc } name [ ( option [, option ...] ) ] VACUUM puts the options befor

Re: [HACKERS] pg_dump's aborted transactions

2015-02-03 Thread Tom Lane
Stephen Frost writes: > All, > We recently had a client complain that check_postgres' commitratio > check would alert about relatively unused databases. As it turns > out, the reason for this is because they automate running pg_dump > against their databases (surely a good thing..), but p

[HACKERS] pg_dump's aborted transactions

2015-02-03 Thread Stephen Frost
All, We recently had a client complain that check_postgres' commitratio check would alert about relatively unused databases. As it turns out, the reason for this is because they automate running pg_dump against their databases (surely a good thing..), but pg_dump doesn't close out its t

Re: [HACKERS] [GENERAL] How do I bump a row to the front of sort efficiently

2015-02-03 Thread BladeOfLight16
On Tue, Feb 3, 2015 at 9:33 PM, BladeOfLight16 wrote: > This is why ORMs are bad. They make hard problems *much* harder, and the > only benefit is that they maybe make easy problems a little quicker. The > cost/savings is *heavily* skewed toward the cost, since there's no upper > bound on the cos

Re: [HACKERS] [GENERAL] How do I bump a row to the front of sort efficiently

2015-02-03 Thread BladeOfLight16
On Mon, Feb 2, 2015 at 1:16 AM, Sam Saffron wrote: > However, the contortions on the above query make it very un-ORM > friendly as I would need to define a view for it but would have no > clean way to pass limits and offsets in. > This is why ORMs are bad. They make hard problems *much* harder,

Re: [HACKERS] Fwd: [GENERAL] 4B row limit for CLOB tables

2015-02-03 Thread David Steele
On 2/3/15 5:27 PM, Jim Nasby wrote: > On 2/3/15 9:50 AM, David Steele wrote: >> EXTENDED is the default for most TOAST-able types and is still subject >> to TOAST_TUPLE_THRESHOLD which is normally 2K. EXTERNAL is the same but >> with no compression. >> >> See: http://www.postgresql.org/docs/9.4/sta

[HACKERS] GRANT USAGE on FOREIGN SERVER exposes passwords

2015-02-03 Thread Noah Yetter
When a user is granted USAGE on a foreign server, the psql command \deu+ will show them the username and password bound to the applicable user mapping. To demonstrate (9.3+): (as a superuser) # create extension postgres_fdw ; # create foreign server loopback_server foreign data wrapper postgres_f

Re: [HACKERS] Re: [COMMITTERS] pgsql: Process 'die' interrupts while reading/writing from the client s

2015-02-03 Thread Andres Freund
On 2015-02-04 00:47:03 +0200, Heikki Linnakangas wrote: > On 02/03/2015 11:51 PM, Andres Freund wrote: > >+ * This is called just after low-level writes. That might be after the read > >+ * finished successfully, or it was interrupted via interrupt. 'blocked' > >tells > >+ * us whether the > > Lo

Re: [HACKERS] Proposal : REINDEX xxx VERBOSE

2015-02-03 Thread Fabrízio de Royes Mello
On Tue, Feb 3, 2015 at 8:26 PM, Jim Nasby wrote: > > On 2/3/15 9:20 AM, Tom Lane wrote: >>> >>> >i.g., I will add following syntax format, >>> >REINDEX ( { INDEX | TABLE | SCHEMA | SYSTEM | DATABASE } , [VERBOSE] ) >>> >name [FORCE]; >> >> Well, the object type is not an optional part of the comma

Re: [HACKERS] Proposal : REINDEX xxx VERBOSE

2015-02-03 Thread Tom Lane
Jim Nasby writes: > On 2/3/15 9:20 AM, Tom Lane wrote: >> Well, the object type is not an optional part of the command. It's >> *necessary*. I was thinking more like >> REINDEX { INDEX | TABLE | etc } name [ ( option [, option ...] ) ] > VACUUM puts the options before the table name, so ISTM it

[HACKERS] Re: [COMMITTERS] pgsql: Process 'die' interrupts while reading/writing from the client s

2015-02-03 Thread Heikki Linnakangas
On 02/03/2015 11:51 PM, Andres Freund wrote: + * This is called just after low-level writes. That might be after the read + * finished successfully, or it was interrupted via interrupt. 'blocked' tells + * us whether the Looks like you forgot to complete that sentence before pushing... Also, n

Re: [HACKERS] [GENERAL] 4B row limit for CLOB tables

2015-02-03 Thread Jim Nasby
On 2/3/15 9:01 AM, Tom Lane wrote: Matthew Kelly writes: However, I do have active databases where the current oid is between 1 billion and 2 billion. They were last dump-restored for a hardware upgrade a couple years ago and were a bit more than half the size. I therefore can imagine that

Re: [HACKERS] Fwd: [GENERAL] 4B row limit for CLOB tables

2015-02-03 Thread Jim Nasby
On 2/3/15 9:50 AM, David Steele wrote: On 2/3/15 10:01 AM, José Luis Tallón wrote: Hmmm alter column set storage external / set storage extended ? From http://www.postgresql.org/docs/9.4/static/sql-altertable.html : ALTER [ COLUMN ] column_name SET STORAGE { PLAIN | EXTERNAL | EXTENDED

Re: [HACKERS] Proposal : REINDEX xxx VERBOSE

2015-02-03 Thread Jim Nasby
On 2/3/15 9:20 AM, Tom Lane wrote: >i.g., I will add following syntax format, >REINDEX ( { INDEX | TABLE | SCHEMA | SYSTEM | DATABASE } , [VERBOSE] ) >name [FORCE]; Well, the object type is not an optional part of the command. It's *necessary*. I was thinking more like REINDEX { INDEX | TABLE

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-03 Thread Josh Berkus
On 02/03/2015 07:50 AM, Robert Haas wrote: > On Tue, Feb 3, 2015 at 10:44 AM, Heikki Linnakangas > wrote: >> That's the whole point of this patch. "max_checkpoint_segments = 10", or >> "max_checkpoint_segments = 160 MB", means that the system will begin a >> checkpoint so that when the checkpoint

Re: [HACKERS] Re: Overhauling our interrupt handling (was Escaping from blocked send() reprised.)

2015-02-03 Thread Andres Freund
On 2015-02-03 22:17:22 +0200, Heikki Linnakangas wrote: > On 02/03/2015 08:54 PM, Andres Freund wrote: > >* Previously the patchset didn't handle SIGTERM while > > InteractiveBackend() was reading from the client. It did handle > > ctrl-c/d, but since getc() isn't interruptible and can't be rep

Re: [HACKERS] Re: Overhauling our interrupt handling (was Escaping from blocked send() reprised.)

2015-02-03 Thread Heikki Linnakangas
On 02/03/2015 08:54 PM, Andres Freund wrote: * Previously the patchset didn't handle SIGTERM while InteractiveBackend() was reading from the client. It did handle ctrl-c/d, but since getc() isn't interruptible and can't be replaced with latches... The fix for that isn't super pretty:

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE} 2.0

2015-02-03 Thread Peter Geoghegan
On Tue, Feb 3, 2015 at 2:05 AM, Heikki Linnakangas wrote: > TRAP: FailedAssertion("!(node->spec != SPEC_INSERT || node->arbiterIndex != > ((Oid) 0))", File: "nodeModifyTable.c", Line: 1619) > > Is that just because of the hack in parse_clause.c? Yes. I never built with assertions and so didn't se

Re: [HACKERS] PQgetssl() and alternative SSL implementations

2015-02-03 Thread Heikki Linnakangas
On 01/28/2015 08:05 PM, Tom Lane wrote: Heikki Linnakangas writes: Right, that was the idea. I wanted it to include the word "OpenSSL", to make it clear in the callers that it's specific to OpenSSL. And SSL, because that's the name of the struct. I agree it looks silly, though. One idea is to h

Re: [HACKERS] Release note bloat is getting out of hand

2015-02-03 Thread Tom Lane
Peter Eisentraut writes: > Note also that you only need to present the release notes from the > latest stable release branch on the web site, as opposed to > documentation for each branch. Yeah, JD suggested the same upthread. If we went over to a separate document containing all the historical

Re: [HACKERS] Getting rid of LSEG.m

2015-02-03 Thread Heikki Linnakangas
On 02/03/2015 06:47 PM, Tom Lane wrote: Or perhaps we should just remove both the field and the ifdef'd assignments. That's a bit more drastic but I can't really see this code ever coming back to life ... especially since the notion of a field that's not stored on disk but is valid in in-memory

[HACKERS] Getting rid of LSEG.m

2015-02-03 Thread Tom Lane
I noticed a Coverity complaint about the "m" field of LSEG being uninitialized, which indeed it is because all the places that would set it are ifdef'd out. So why didn't the field itself get ifdef'd out as well? Or perhaps we should just remove both the field and the ifdef'd assignments. That's

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-03 Thread Petr Jelinek
On 03/02/15 16:50, Robert Haas wrote: On Tue, Feb 3, 2015 at 10:44 AM, Heikki Linnakangas wrote: That's the whole point of this patch. "max_checkpoint_segments = 10", or "max_checkpoint_segments = 160 MB", means that the system will begin a checkpoint so that when the checkpoint completes, and

Re: [HACKERS] Release note bloat is getting out of hand

2015-02-03 Thread Peter Eisentraut
On 2/3/15 12:32 AM, Tom Lane wrote: > It would not make that much of a difference in tarball size, agreed. > It *would* make a difference in the build time and output size of the > SGML docs --- as I mentioned at the outset, the release notes currently > account for 25% of the SGML source linecount

Re: [HACKERS] Release note bloat is getting out of hand

2015-02-03 Thread Tom Lane
Peter Eisentraut writes: > On 2/3/15 10:11 AM, Marko Tiikkaja wrote: >> And now that we're on the subject of ponies, it would be nice if the >> relevant git hashes were included as well. > That's probably not going to happen. A release-note entry is often the > combination of many commits, and a

Re: [HACKERS] Add LINE: hint when schemaname.typename is a non-existent schema

2015-02-03 Thread Tom Lane
Ryan Kelly writes: > The attached patch adds a LINE: ... hint when schemaname.typename > results in a schema which does not exist. I came across this when a > missing comma in a SELECT list resulted in an error without a location > in a query a few thousand lines long. I think this is a good prob

Re: [HACKERS] Getting rid of wal_level=archive and default to hot_standby + wal_senders

2015-02-03 Thread Robert Haas
On Tue, Feb 3, 2015 at 7:43 AM, Andres Freund wrote: > I think these days there's no reason for the split between the archive > and hot_standby wal levels. The split was made out of volume and > stability concerns. I think we can by now be confident about the > wal_level = hot_standby changes (not

Re: [HACKERS] Release note bloat is getting out of hand

2015-02-03 Thread Peter Eisentraut
On 2/3/15 10:11 AM, Marko Tiikkaja wrote: > And now that we're on the subject of ponies, it would be nice if the > relevant git hashes were included as well. That's probably not going to happen. A release-note entry is often the combination of many commits, and accurately tracking those is a lot

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-03 Thread Robert Haas
On Tue, Feb 3, 2015 at 10:44 AM, Heikki Linnakangas wrote: >>> Works for me. However, note that "max_checkpoint_segments = 10" doesn't >>> mean >>> the same as current "checkpoint_segments = 10". With checkpoint_segments >>> = >>> 10 you end up with about 2x-3x as much WAL as with >>> max_checkpoi

Re: [HACKERS] Fwd: [GENERAL] 4B row limit for CLOB tables

2015-02-03 Thread David Steele
On 2/3/15 10:01 AM, José Luis Tallón wrote: > Hmmm alter column set storage external / set storage extended ? > > From http://www.postgresql.org/docs/9.4/static/sql-altertable.html : > ALTER [ COLUMN ] column_name SET STORAGE { PLAIN | EXTERNAL | > EXTENDED | MAIN } > > This would do w

Re: [HACKERS] Proposal : REINDEX xxx VERBOSE

2015-02-03 Thread Tom Lane
Andres Freund writes: > On 2015-02-03 10:20:03 -0500, Tom Lane wrote: >> Well, the object type is not an optional part of the command. It's >> *necessary*. I was thinking more like >> >> REINDEX { INDEX | TABLE | etc } name [ ( option [, option ...] ) ] >> >> option := FORCE | VERBOSE >> >> W

Re: [HACKERS] Getting rid of wal_level=archive and default to hot_standby + wal_senders

2015-02-03 Thread Andres Freund
On 2015-02-03 10:41:04 -0500, Tom Lane wrote: > Andres Freund writes: > > Additionally I think we should change the default for wal_level to > > hot_standby and max_wal_senders (maybe to 5). That way users can use > > pg_basebackup and setup streaming standbys without having to restart the > > pri

Re: [HACKERS] Add LINE: hint when schemaname.typename is a non-existent schema

2015-02-03 Thread Robert Haas
On Mon, Feb 2, 2015 at 2:44 PM, Ryan Kelly wrote: > The attached patch adds a LINE: ... hint when schemaname.typename > results in a schema which does not exist. I came across this when a > missing comma in a SELECT list resulted in an error without a location > in a query a few thousand lines lon

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-03 Thread Heikki Linnakangas
On 02/03/2015 05:19 PM, Robert Haas wrote: On Tue, Feb 3, 2015 at 7:31 AM, Heikki Linnakangas wrote: On 02/02/2015 03:36 PM, Robert Haas wrote: Second, I*think* that these settings are symmetric and, if that's right, then I suggest that they ought to be named symmetrically. Basically, I think

Re: [HACKERS] Getting rid of wal_level=archive and default to hot_standby + wal_senders

2015-02-03 Thread Tom Lane
Andres Freund writes: > Additionally I think we should change the default for wal_level to > hot_standby and max_wal_senders (maybe to 5). That way users can use > pg_basebackup and setup streaming standbys without having to restart the > primary. I think that'd be a important step in making setup

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2015-02-03 Thread Robert Haas
On Mon, Feb 2, 2015 at 9:10 AM, Andres Freund wrote: > On 2014-11-12 16:11:58 -0500, Robert Haas wrote: >> On Wed, Nov 12, 2014 at 4:10 PM, Robert Haas wrote: >> > On Thu, Nov 6, 2014 at 9:50 AM, Peter Eisentraut wrote: >> >> If REINDEX cannot work without an exclusive lock, we should invent som

Re: [HACKERS] Unnecessary pointer-NULL checks in pgp-pgsql.c

2015-02-03 Thread Robert Haas
On Sun, Feb 1, 2015 at 9:14 PM, Michael Paquier wrote: > Coverity is pointing out that we are doing pointer-NULL checks on > things that cannot be NULL in decrypt_internal(): > out: > - if (src) > - mbuf_free(src); > - if (ctx) > - pgp_free(ctx); > +

Re: [HACKERS] Proposal : REINDEX xxx VERBOSE

2015-02-03 Thread Andres Freund
On 2015-02-03 10:20:03 -0500, Tom Lane wrote: > Sawada Masahiko writes: > > On Tue, Feb 3, 2015 at 12:32 AM, Tom Lane wrote: > >> The way that FORCE was added to REINDEX was poorly thought out; let's not > >> double down on that with another option added without any consideration > >> for future

Re: [HACKERS] Proposal : REINDEX xxx VERBOSE

2015-02-03 Thread Tom Lane
Sawada Masahiko writes: > On Tue, Feb 3, 2015 at 12:32 AM, Tom Lane wrote: >> The way that FORCE was added to REINDEX was poorly thought out; let's not >> double down on that with another option added without any consideration >> for future expansion. I'd be happier if we adopted something simil

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-03 Thread Robert Haas
On Tue, Feb 3, 2015 at 7:31 AM, Heikki Linnakangas wrote: > On 02/02/2015 03:36 PM, Robert Haas wrote: >> Second, I*think* that these settings are symmetric and, if that's >> right, then I suggest that they ought to be named symmetrically. >> Basically, I think you've got min_checkpoint_segments

Re: [HACKERS] How about to have relnamespace and relrole?

2015-02-03 Thread Tom Lane
Kyotaro HORIGUCHI writes: > Most of OID types has reg* OID types. Theses are very convenient > when looking into system catalog/views, but there aren't OID > types for userid and namespace id. > What do you think about having these new OID types? I'm not really excited about that. That line of

Re: [HACKERS] Release note bloat is getting out of hand

2015-02-03 Thread Marko Tiikkaja
On 2/3/15 4:04 PM, David Fetter wrote: On Mon, Feb 02, 2015 at 08:56:19PM -, Greg Sabino Mullane wrote: Robert Haas wrote: but there are times when it's easier to find out what release introduced a feature by looking at the release notes, and it's certainly more useful if you want to send a

Re: [HACKERS] Release note bloat is getting out of hand

2015-02-03 Thread David Fetter
On Mon, Feb 02, 2015 at 08:56:19PM -, Greg Sabino Mullane wrote: > Robert Haas wrote: > > but there are times when it's easier to find out what release > > introduced a feature by looking at the release notes, and it's > > certainly more useful if you want to send a link to someone who > > i

Re: [HACKERS] [GENERAL] 4B row limit for CLOB tables

2015-02-03 Thread Tom Lane
Matthew Kelly writes: > However, I do have active databases where the current oid is between 1 > billion and 2 billion. They were last dump-restored for a hardware upgrade a > couple years ago and were a bit more than half the size. I therefore can > imagine that I have tables which are keyed

Re: [HACKERS] Fwd: [GENERAL] 4B row limit for CLOB tables

2015-02-03 Thread José Luis Tallón
On 02/03/2015 03:44 AM, Jim Nasby wrote: [snip] The alternative could be some "long LOB" ("HugeOBject"?) using the equivalent to "serial8" whereas regular LOBs would use "serial4". Well, it depends on how we did this. We could (for example) add a field to pg_class that determines what type to

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-03 Thread Kevin Grittner
Heikki Linnakangas wrote: > On 02/02/2015 04:21 PM, Andres Freund wrote: >> On 2015-02-02 08:36:41 -0500, Robert Haas wrote: >>> Also, I'd like to propose that we set the default value of >>> max_checkpoint_segments/checkpoint_wal_size to something at >>> least an order of magnitude larger than th

Re: [HACKERS] Release note bloat is getting out of hand

2015-02-03 Thread David Fetter
On Tue, Feb 03, 2015 at 09:08:45AM -0500, Andrew Dunstan wrote: > > On 02/03/2015 08:55 AM, Kevin Grittner wrote: > >Tom Lane wrote: > >>Josh Berkus writes: > >>>On 02/02/2015 05:39 PM, Peter Eisentraut wrote: > I share the sentiment that the release notes *seem* too big, but the > subse

Re: [HACKERS] Release note bloat is getting out of hand

2015-02-03 Thread Magnus Hagander
On Tue, Feb 3, 2015 at 3:51 PM, Robert Haas wrote: > On Mon, Feb 2, 2015 at 4:07 PM, Tom Lane wrote: > > Robert Haas writes: > >> On Mon, Feb 2, 2015 at 3:11 PM, Tom Lane wrote: > >>> So I'm bemused by Robert's insistence that he wants that format to > support > >>> searches. As I said, I fin

Re: [HACKERS] Release note bloat is getting out of hand

2015-02-03 Thread Robert Haas
On Mon, Feb 2, 2015 at 4:07 PM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Feb 2, 2015 at 3:11 PM, Tom Lane wrote: >>> So I'm bemused by Robert's insistence that he wants that format to support >>> searches. As I said, I find it far more convenient to search the output >>> of "git log" an

Re: [HACKERS] Proposal : REINDEX xxx VERBOSE

2015-02-03 Thread Fabrízio de Royes Mello
On Tue, Feb 3, 2015 at 9:09 AM, Sawada Masahiko wrote: > > On Tue, Feb 3, 2015 at 12:32 AM, Tom Lane wrote: > > Sawada Masahiko writes: > >> On Mon, Feb 2, 2015 at 9:21 PM, Michael Paquier > >> wrote: > >>> Now, I think that it may > >>> be better to provide the keyword VERBOSE before the type

Re: [HACKERS] Release note bloat is getting out of hand

2015-02-03 Thread Andrew Dunstan
On 02/03/2015 08:55 AM, Kevin Grittner wrote: Tom Lane wrote: Josh Berkus writes: On 02/02/2015 05:39 PM, Peter Eisentraut wrote: I share the sentiment that the release notes *seem* too big, but the subsequent discussion shows that it's not clear why that's really a problem. Exactly what p

Re: [HACKERS] Release note bloat is getting out of hand

2015-02-03 Thread Kevin Grittner
Tom Lane wrote: > Josh Berkus writes: >> On 02/02/2015 05:39 PM, Peter Eisentraut wrote: >>> I share the sentiment that the release notes *seem* too big, but the >>> subsequent discussion shows that it's not clear why that's really a >>> problem. Exactly what problem are we trying to fix? >> >>

Re: [HACKERS] Unlikely-to-happen crash in ecpg driver caused by NULL-pointer check not done

2015-02-03 Thread Michael Paquier
On Tue, Feb 3, 2015 at 7:50 PM, Heikki Linnakangas wrote: > Hmm. Since the ecpg_add_mem call is done after setting (*(void **) var), > that's left to point to already-free'd memory. The other call sites have a > similar issue. I haven't analyzed the code to check if that's harmless or > not, but s

Re: [HACKERS] Getting rid of wal_level=archive and default to hot_standby + wal_senders

2015-02-03 Thread Andres Freund
On 2015-02-03 21:58:44 +0900, Michael Paquier wrote: > On Tue, Feb 3, 2015 at 9:43 PM, Andres Freund wrote: > > I think these days there's no reason for the split between the archive > > and hot_standby wal levels. The split was made out of volume and > > stability concerns. I think we can by now

Re: [HACKERS] Getting rid of wal_level=archive and default to hot_standby + wal_senders

2015-02-03 Thread Petr Jelinek
On 03/02/15 13:51, Magnus Hagander wrote: On Tue, Feb 3, 2015 at 1:43 PM, Andres Freund mailto:and...@2ndquadrant.com>> wrote: Hi, I think these days there's no reason for the split between the archive and hot_standby wal levels. The split was made out of volume and stability co

Re: [HACKERS] Getting rid of wal_level=archive and default to hot_standby + wal_senders

2015-02-03 Thread Andres Freund
On 2015-02-03 13:51:25 +0100, Magnus Hagander wrote: > Those who want to optimize their WAL size can set it back to minimal, but > let's make the default the one that makes life *easy* for people. Precisely. New users won't usually have tremendous stuff to load in the specific circumstances in whi

Re: [HACKERS] Getting rid of wal_level=archive and default to hot_standby + wal_senders

2015-02-03 Thread Michael Paquier
On Tue, Feb 3, 2015 at 9:43 PM, Andres Freund wrote: > I think these days there's no reason for the split between the archive > and hot_standby wal levels. The split was made out of volume and > stability concerns. I think we can by now be confident about the > wal_level = hot_standby changes (not

Re: [HACKERS] Getting rid of wal_level=archive and default to hot_standby + wal_senders

2015-02-03 Thread Magnus Hagander
On Tue, Feb 3, 2015 at 1:43 PM, Andres Freund wrote: > Hi, > > I think these days there's no reason for the split between the archive > and hot_standby wal levels. The split was made out of volume and > stability concerns. I think we can by now be confident about the > wal_level = hot_standby cha

Re: [HACKERS] Small memory leak in execute.c of ECPG driver

2015-02-03 Thread Michael Paquier
On Tue, Feb 3, 2015 at 5:35 PM, Heikki Linnakangas wrote: > I think there are more similar leaks nearby. After the first hunk, there's > another if-check with "return false" that also leaks mallocedval. Right > after the two other hunks, if the ecpg_realloc fails, we again leak > mallocedval. Yes,

[HACKERS] Getting rid of wal_level=archive and default to hot_standby + wal_senders

2015-02-03 Thread Andres Freund
Hi, I think these days there's no reason for the split between the archive and hot_standby wal levels. The split was made out of volume and stability concerns. I think we can by now be confident about the wal_level = hot_standby changes (note I'm not proposing hot_standby = on). So let's remove t

Re: [HACKERS] SSL information view

2015-02-03 Thread Magnus Hagander
On Tue, Feb 3, 2015 at 6:42 AM, Michael Paquier wrote: > Where are we on this patch? No new version has been provided and there > have been comments provided by Heikki here > (5491e547.4040...@vmware.com) and by Alexei here > (87ppbqz00h@commandprompt.com). > > Yeah, it's on my shame list. I

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-03 Thread Heikki Linnakangas
On 02/02/2015 03:36 PM, Robert Haas wrote: Second, I*think* that these settings are symmetric and, if that's right, then I suggest that they ought to be named symmetrically. Basically, I think you've got min_checkpoint_segments (the number of recycled segments we keep around always) and max_chec

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-03 Thread Heikki Linnakangas
On 02/02/2015 04:21 PM, Andres Freund wrote: Hi, On 2015-02-02 08:36:41 -0500, Robert Haas wrote: Also, I'd like to propose that we set the default value of max_checkpoint_segments/checkpoint_wal_size to something at least an order of magnitude larger than the current default setting. +1 I

Re: [HACKERS] PageRepairFragmentation performance

2015-02-03 Thread Heikki Linnakangas
On 01/31/2015 01:49 AM, Peter Geoghegan wrote: The refactoring patch certainly looks very reasonable. Ok, committed the refactoring part for now. Thanks for the review. - Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http:

Re: [HACKERS] Fetch zero result rows when executing a query?

2015-02-03 Thread Andres Freund
Hi, On 2015-02-03 12:26:33 +0100, Shay Rojansky wrote: > Sorry if this has been asked before, couldn't find any mention... > > I'm working on the Npgsql, the .NET driver for PostgreSQL, and am trying to > find a way to execute a query but without fetching any rows. The Execute > message has a max

[HACKERS] Fetch zero result rows when executing a query?

2015-02-03 Thread Shay Rojansky
Sorry if this has been asked before, couldn't find any mention... I'm working on the Npgsql, the .NET driver for PostgreSQL, and am trying to find a way to execute a query but without fetching any rows. The Execute message has a maximum result-row count, but zero is documented to mean "fetch all r

Re: [HACKERS] Proposal : REINDEX xxx VERBOSE

2015-02-03 Thread Sawada Masahiko
On Tue, Feb 3, 2015 at 12:32 AM, Tom Lane wrote: > Sawada Masahiko writes: >> On Mon, Feb 2, 2015 at 9:21 PM, Michael Paquier >> wrote: >>> Now, I think that it may >>> be better to provide the keyword VERBOSE before the type of object >>> reindexed as REINDEX [ VERBOSE ] object. > >> Actually,

Re: [HACKERS] Unlikely-to-happen crash in ecpg driver caused by NULL-pointer check not done

2015-02-03 Thread Heikki Linnakangas
On 02/03/2015 09:28 AM, Michael Paquier wrote: Hi all, In ecpg_add_mem of memory.c, we use ecpg_alloc but there is actually no NULL-pointer check. If an OOM shows up exactly at this point, this is likely to cause a crash. Attached patch adds some extra processing to ecpg_add_mem to check if the

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE} 2.0

2015-02-03 Thread Heikki Linnakangas
On 01/30/2015 01:38 AM, Peter Geoghegan wrote: On the stress-testing front, I'm still running Jeff Janes' tool [1], while also continuing to use his Postgres modifications to artificially increase the XID burn rate. [1] https://github.com/petergeoghegan/jjanes_upsert I followed the instruction

Re: [HACKERS] File based Incremental backup v9

2015-02-03 Thread Marco Nenciarini
Il 02/02/15 22:28, Magnus Hagander ha scritto: > On Mon, Feb 2, 2015 at 10:06 PM, Robert Haas > wrote: > > On Sat, Jan 31, 2015 at 6:47 PM, Marco Nenciarini > > wrote: > > Il 31/01/15 17:22, Erik Rijkers ha scr

[HACKERS] How about to have relnamespace and relrole?

2015-02-03 Thread Kyotaro HORIGUCHI
Hello, Most of OID types has reg* OID types. Theses are very convenient when looking into system catalog/views, but there aren't OID types for userid and namespace id. What do you think about having these new OID types? The usefulness of regnamespace is doubtful but regrole should be useful becau

Re: [HACKERS] Hot Standby WAL reply uses heavyweight session locks, but doesn't have enough infrastructure set up

2015-02-03 Thread Andres Freund
On 2015-02-03 14:18:02 +0900, Michael Paquier wrote: > - "RecoveryLockList contains entry for lock no longer recorded by > lock manager: xid %u database %u relation %u", > - lock->xid, lock->dbOid, lock->relOid); > +"RecoveryLockList contains entry for lock no longer recorded by > loc

Re: [HACKERS] Performance improvement for joins where outer side is unique

2015-02-03 Thread Kyotaro HORIGUCHI
Hi, I had a look on this patch. Although I haven't understood whole the stuff and all of the related things, I will comment as possible. Performance: I looked on the performance gain this patch gives. For several on-memory joins, I had gains about 3% for merge join, 5% for hash join, and 10% for

[HACKERS] Performance difference between kernel 3.10 and 3.12

2015-02-03 Thread Tatsuo Ishii
I see significant performance difference between systems using Linux kernel 3.10 and 3.12. We did tests on CentOS7 (kernel 3.10) and SuSE Enterprise Linux Server 12 (kernel 3.12). Both systems run on VMWare ESXi 5.5.0. Xeon CPU E52650 2GHz x 2 (16 cores in total) Mem 96GB, 10,000 RPM SAS drive Gu

Re: [HACKERS] Missing markup in pg_receivexlog.sgml

2015-02-03 Thread Heikki Linnakangas
On 02/03/2015 10:27 AM, Michael Paquier wrote: On Tue, Feb 3, 2015 at 4:25 PM, Heikki Linnakangas wrote: Flush the WAL data to disk immediately after it's being received. Also send a status packet back to the server immediately after flushing, regardless of --status-interval Yes, that's indeed

Re: [HACKERS] [GENERAL] 4B row limit for CLOB tables

2015-02-03 Thread Matthew Kelly
> Hmm 2^32 times aprox. 2kB (as per usual heuristics, ~4 rows per heap > page) is 8796093022208 (~9e13) bytes > ... which results in 8192 1GB segments :O 8192 1GB segments is just 8TB, its not _that_ large. At TripAdvisor we’ve been using a NoSQL solution to do session storage. We are look

Re: [HACKERS] Small memory leak in execute.c of ECPG driver

2015-02-03 Thread Heikki Linnakangas
On 02/03/2015 08:58 AM, Michael Paquier wrote: Hi all, In exactly 3 places of the ECPG driver (for numeric, for interval and for date), we do something as follows: /* Allocation of mallocedval */ if (!(mallocedval = ecpg_strdup("array [", lineno))) return false; for (element = 0; eleme

Re: [HACKERS] Missing markup in pg_receivexlog.sgml

2015-02-03 Thread Michael Paquier
On Tue, Feb 3, 2015 at 4:25 PM, Heikki Linnakangas wrote: > Flush the WAL data to disk immediately after it's being received. Also send > a status packet back to the server immediately after flushing, regardless of > --status-interval Yes, that's indeed better. As long as I am on it, attached is a