Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-11-18 Thread Haribabu kommi
On 18 November 2013 23:25 Andres Freund wrote: > On 2013-11-18 15:01:42 +, Haribabu kommi wrote: > > > > /* > > + * Returns the malloced string of containing current working > directory. > > + * The caller has to take care of freeing the memory. > > + * On failure exits with error code. > > +

Re: [HACKERS] Improvement of pg_stat_statement usage about buffer hit ratio

2013-11-18 Thread Peter Geoghegan
On Mon, Nov 18, 2013 at 10:08 PM, KONDO Mitsumasa wrote: > I regret past decision of Itagaki-san's patch, and improvement might not be > possible. However, we can change it, if we get have logical reason to change > it. I suppose so. >> I don't think that the tool needs to be expensive. If selec

[HACKERS] Large offset optimization and index only scan

2013-11-18 Thread Maxim Boguk
Hi, Recently, I found very interesting case where using IOS could provide serious benefits, but database not able to use it. I think my ideas could be beneficial in two ways: 1)as work around of the current limitations 2)as idea to improve planner/executor in the future Case: SELECT * FROM very

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-18 Thread KONDO Mitsumasa
(2013/11/15 19:27), Sawada Masahiko wrote: On Thu, Nov 14, 2013 at 7:51 PM, Florian Weimer wrote: On 11/14/2013 07:02 AM, Sawada Masahiko wrote: I attached patch adds new wal_level 'all'. Shouldn't this be a separate setting? It's useful for storage which requires rewriting a partially wr

Re: [HACKERS] lock on object is already held

2013-11-18 Thread Tom Lane
Daniel Wood writes: > Sorry but I don't know how to respond to an old thread I found on > postgresql.org: > http://www.postgresql.org/message-id/20766.1357318...@sss.pgh.pa.us > I presume this is still an open issue. While working on a new feature I > wrote a stress test for it. After fixing my

Re: [HACKERS] Improvement of pg_stat_statement usage about buffer hit ratio

2013-11-18 Thread KONDO Mitsumasa
(2013/11/19 12:03), Peter Geoghegan wrote: On Mon, Nov 18, 2013 at 6:12 PM, KONDO Mitsumasa wrote: I confirmed that Itagaki-san and Mr Cerdic disscution. He said that raw values be just simple. However, were his changes just simple? I cannot understand his aesthetics sense and also you, too:-(

Re: [HACKERS] pre-commit triggers

2013-11-18 Thread Noah Misch
On Fri, Nov 15, 2013 at 01:01:48PM -0500, Andrew Dunstan wrote: > The triggers don't fire if there is no real XID, so only actual data > changes should cause the trigger to fire. What's the advantage of this provision? Without it, an individual trigger could make the same check and drop out qui

Re: [HACKERS] shared memory message queues

2013-11-18 Thread Kohei KaiGai
Hello, I tried to look at the patch #1 and #2 at first, but I shall rest of portion later. * basic checks All the patches (not only #1, #2) can be applied without any problem towards the latest master branch. Its build was succeeded with Werror. Regression test works fine on the core and contrib/

Re: [HACKERS] COPY table FROM STDIN doesn't show count tag

2013-11-18 Thread Amit Khandekar
On 18 November 2013 18:00, Rajeev rastogi wrote: > On 18 November 2013, Amit Khandekar wrote: > > >> On 18 October 2013 17:07, Rajeev rastogi > wrote: > > >>From the following mail, copy behaviour between stdin and normal file > having some inconsistency. > > >> > http://www.postgresql.org/mess

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-11-18 Thread Amit Kapila
On Fri, Nov 15, 2013 at 10:52 AM, Haribabu kommi wrote: > On 15 November 2013 10:00 Amit Kapila wrote: >> On Wed, Nov 13, 2013 at 12:02 PM, Haribabu kommi >> wrote: >> > >> > With the simulated bloat test run for 1 hour the bloat occurred as >> > below, >> > >> > Unpatched - 1532MB >> > Patched

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-11-18 Thread Amit Kapila
On Mon, Nov 18, 2013 at 8:31 PM, Haribabu kommi wrote: > On 18 November 2013 20:01 Amit Kapila wrote: >> > Code changes are fine. >> > If two or three errors are present in the configuration file, it >> prints the last error >> > Configuration parameter file only. Is it required to be mentioned in

[HACKERS] lock on object is already held

2013-11-18 Thread Daniel Wood
Sorry but I don't know how to respond to an old thread I found on postgresql.org: http://www.postgresql.org/message-id/20766.1357318...@sss.pgh.pa.us I presume this is still an open issue. While working on a new feature I wrote a stress test for it. After fixing my bugs, I couldn't get rid of: E

Re: [HACKERS] Re: Using indexes for ORDER BY and PARTITION BY clause in windowing functions

2013-11-18 Thread Sameer Kumar
On Thu, Nov 14, 2013 at 6:51 PM, Kyotaro HORIGUCHI < horiguchi.kyot...@lab.ntt.co.jp> wrote: > Hello, > > > When I read it again and try to relate, I get your point. Actually true, > > hashes must always be performed as last option (if that is what you too > > meant) and if there are few other ope

[HACKERS] Re: Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-18 Thread David Johnston
Bruce Momjian wrote > On Mon, Nov 18, 2013 at 06:30:32PM -0800, David Johnston wrote: >> > Personally, I am fine with changing them all to WARNING. >> >> Error makes more sense if the goal is internal consistency. That goal >> should be subservient to backward compatibility. Changing LOCK to >>

Re: [HACKERS] LISTEN / NOTIFY enhancement request for Postgresql

2013-11-18 Thread Sev Zaslavsky
Thank you all for considering my feature request. Dimitri's suggestion is a very good one - I feel it will accomplish the goal of allowing more granularity in the "Listen". We might also want to add a flag in postgresql.conf to disable this enhancement so that we don't break existing code.

Re: [HACKERS] COPY table FROM STDIN doesn't show count tag

2013-11-18 Thread Rajeev rastogi
On 18 November 2013, Amit Khandekar wrote: >> On 18 October 2013 17:07, Rajeev rastogi >> mailto:rajeev.rast...@huawei.com>> wrote: >>From the following mail, copy behaviour between stdin and normal file having >>some inconsistency. >> >> http://www.postgresql.org/message-id/ce85a517.4878e

[HACKERS]

2013-11-18 Thread Rajeev rastogi
On 18 November 2013, Amit Khandekar wrote: >> On 18 October 2013 17:07, Rajeev rastogi >> mailto:rajeev.rast...@huawei.com>> wrote: >>From the following mail, copy behaviour between stdin and normal file having >>some inconsistency. >> >> http://www.postgresql.org/message-id/ce85a517.4878

Re: [HACKERS] Re: Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-18 Thread Bruce Momjian
On Mon, Nov 18, 2013 at 06:30:32PM -0800, David Johnston wrote: > > Personally, I am fine with changing them all to WARNING. > > Error makes more sense if the goal is internal consistency. That goal > should be subservient to backward compatibility. Changing LOCK to warning > is less problematic

Re: [HACKERS] pg_upgrade: delete_old_cluster.sh issues

2013-11-18 Thread Bruce Momjian
On Tue, Nov 12, 2013 at 10:35:58AM +, Marc Mamin wrote: > Hello, > > IMHO, there is a serious issue in the script to clean the old data directory > when running pg_upgrade in link mode. > > in short: When working with symbolic links, the first step in > delete_old_cluster.sh > is to delete

Re: [HACKERS] GIN improvements part2: fast scan

2013-11-18 Thread Rod Taylor
I tried again this morning using gin-packed-postinglists-16.patch and gin-fast-scan.6.patch. No crashes. It is about a 0.1% random sample of production data (10,000,000 records) with the below structure. Pg was compiled with debug enabled in both cases. Table "public.kp" Column | Type |

Re: [HACKERS] GIN improvements part2: fast scan

2013-11-18 Thread Rod Taylor
I checked out master and put together a test case using a small percentage of production data for a known problem we have with Pg 9.2 and text search scans. A small percentage in this case means 10 million records randomly selected; has a few billion records. Tests ran for master successfully an

Re: [HACKERS] Add \i option to bring in the specified file as a quoted literal

2013-11-18 Thread Piotr Marcinczyk
Dnia 2013-11-13, śro o godzinie 10:26 -0500, Bruce Momjian pisze: > On Wed, Nov 13, 2013 at 08:58:07AM +0530, Amit Kapila wrote: > > On Tue, Nov 12, 2013 at 9:37 PM, Bruce Momjian wrote: > > > On Wed, Oct 23, 2013 at 10:31:39AM +0530, Amit Kapila wrote: > > >> On Tue, Oct 22, 2013 at 3:04 AM, Piot

[HACKERS] Standalone synchronous master

2013-11-18 Thread Rajeev rastogi
This patch implements the following TODO item: Add a new "eager" synchronous mode that starts out synchronous but reverts to asynchronous after a failure timeout period This would require some type of command to be executed to alert administrators of this change. http://archives.postgresql.org/p

Re: [HACKERS] Improvement of pg_stat_statement usage about buffer hit ratio

2013-11-18 Thread Peter Geoghegan
On Mon, Nov 18, 2013 at 6:12 PM, KONDO Mitsumasa wrote: > I confirmed that Itagaki-san and Mr Cerdic disscution. He said that raw > values be just simple. However, were his changes just simple? I cannot > understand his aesthetics sense and also you, too:-( > It's too complicated, and do you know

Re: [HACKERS] Transaction-lifespan memory leak with plpgsql DO blocks

2013-11-18 Thread Dilip kumar
On 13 November 2013 03:17 David Johnston wrote, > > Having had this same thought WRT the "FOR UPDATE in LOOP" bug posting > the lack of a listing of outstanding bugs does leave some gaps. I > would imagine people would appreciate something like: > > Frequency: Rare > Severity: Low > Fix Complex

[HACKERS] Remove use of obsolescent Autoconf macros

2013-11-18 Thread Peter Eisentraut
According to the Autoconf documentation, the following macros are obsolescent because no reasonably current system has the defect that they are testing for. - AC_C_CONST - AC_C_STRINGIZE - AC_C_VOLATILE - AC_FUNC_MEMCMP Therefore, I propose to remove their use, with the attached patch. >From 21d

Re: [HACKERS] Improvement of pg_stat_statement usage about buffer hit ratio

2013-11-18 Thread KONDO Mitsumasa
(2013/11/19 11:12), KONDO Mitsumasa wrote: (2013/11/19 3:56), Peter Geoghegan wrote: On Mon, Nov 18, 2013 at 10:49 AM, Fujii Masao wrote: The same idea was proposed before but not committed because Itagaki thought that pg_stat_statements view should report only raw values. Please read the foll

[HACKERS] Re: Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-18 Thread David Johnston
Bruce Momjian wrote > On Mon, Nov 18, 2013 at 05:05:45PM -0800, David Johnston wrote: >> Bruce Momjian wrote >> > Considering we are doing this outside of a transaction, and WARNING or >> > ERROR is pretty much the same, from a behavioral perspective. >> > >> > Should we change this and LOCK to be

Re: [HACKERS] [PATCH] configure: allow adding a custom string to PG_VERSION

2013-11-18 Thread Michael Paquier
On Tue, Nov 19, 2013 at 10:48 AM, Peter Eisentraut wrote: > I think this is a reasonable feature, and the implementation is OK, but > I don't see why the format of the extra version information needs to be > exactly > > PG_VERSION="$PACKAGE_VERSION ($withval)" > > I'd imagine, for example, tha

Re: [HACKERS] Re: Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-18 Thread Bruce Momjian
On Mon, Nov 18, 2013 at 05:05:45PM -0800, David Johnston wrote: > Bruce Momjian wrote > > Considering we are doing this outside of a transaction, and WARNING or > > ERROR is pretty much the same, from a behavioral perspective. > > > > Should we change this and LOCK to be a warning? > > >From the

Re: [HACKERS] Improvement of pg_stat_statement usage about buffer hit ratio

2013-11-18 Thread KONDO Mitsumasa
(2013/11/19 3:56), Peter Geoghegan wrote: On Mon, Nov 18, 2013 at 10:49 AM, Fujii Masao wrote: The same idea was proposed before but not committed because Itagaki thought that pg_stat_statements view should report only raw values. Please read the following thread. I have the same feeling with h

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2013-11-18 Thread Peter Geoghegan
On Mon, Nov 18, 2013 at 4:37 PM, Peter Geoghegan wrote: > You're right that the value locking is scary. I think we need to very > carefully consider it, once I have buy-in on the basic approach. I > really do think it's the least-worst approach described to date. It > isn't like we can't discuss m

Re: [HACKERS] [PATCH] configure: allow adding a custom string to PG_VERSION

2013-11-18 Thread Peter Eisentraut
On Tue, 2013-11-05 at 18:29 +0200, Oskari Saarenmaa wrote: > This can be used to tag custom built packages with an extra version string > such as the git describe id or distribution package release version. I think this is a reasonable feature, and the implementation is OK, but I don't see why the

Re: [HACKERS] More legacy code: pg_ctl

2013-11-18 Thread Josh Berkus
On 11/18/2013 05:09 PM, Josh Berkus wrote: > Folks, > > Speaking of legacy code with bad default behaviors: pg_ctl. The current > utility is designed to fathfully reproduce the rather hackish shell > script we originally had for postgres startup. This results in a > couple of unintuitive defaul

[HACKERS] More legacy code: pg_ctl

2013-11-18 Thread Josh Berkus
Folks, Speaking of legacy code with bad default behaviors: pg_ctl. The current utility is designed to fathfully reproduce the rather hackish shell script we originally had for postgres startup. This results in a couple of unintuitive defaults which give sysadmins and config management developer

[HACKERS] Re: Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-18 Thread David Johnston
Bruce Momjian wrote > Considering we are doing this outside of a transaction, and WARNING or > ERROR is pretty much the same, from a behavioral perspective. > > Should we change this and LOCK to be a warning? >From the calling application's perspective an error and a warning are definitely behavi

Re: [HACKERS] GIN improvements part2: fast scan

2013-11-18 Thread Rod Taylor
On Fri, Nov 15, 2013 at 2:42 PM, Alexander Korotkov wrote: > On Fri, Nov 15, 2013 at 11:39 PM, Rod Taylor wrote: > >> >> The patched index is 58% of the 9.4 master size. 212 MB instead of 365 MB. >> > > Good. That's meet my expectations :) > You mention that both master and patched versions was c

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2013-11-18 Thread Peter Geoghegan
On Mon, Nov 18, 2013 at 6:44 AM, Heikki Linnakangas wrote: > I think it's important to recap the design goals of this. Seems reasonable to list them out. > * It should be usable and perform well for both large batch updates and > small transactions. I think that that's a secondary goal, a quest

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2013-11-18 Thread Jim Mlodgenski
On Mon, Nov 18, 2013 at 7:25 AM, Kohei KaiGai wrote: > The attached patches are the revised custom-scan APIs. > My initial review on this feature: - The patches apply and build, but it produces a warning: ctidscan.c: In function ‘CTidInitCustomScanPlan’: ctidscan.c:362:9: warning: unused variabl

[HACKERS] Review: HStore Gin Speedup

2013-11-18 Thread Antonin Houska
Following are my initial comments for https://commitfest.postgresql.org/action/patch_view?id=1203 1. It no longer applies, probably due to commit a18167510f4c385329697588ce5132cbf95779c3 error: contrib/hstore/hstore--1.1.sql: No such file or directory 2. Compatibility with HStore v2.0 (https:/

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 2:26 PM, Andres Freund wrote: > Trying to reproduce the issue with and without hot_standby=on would be > very helpful, but I guess that's time consuming? I've been working on it, but I haven't gotten it to fail yet. I'll keep at it. -- -- Christophe Pettus x...@thebuild

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Andres Freund
On 2013-11-18 14:25:58 -0800, Christophe Pettus wrote: > Great! If there's any further data I can supply to help, let me know. Trying to reproduce the issue with and without hot_standby=on would be very helpful, but I guess that's time consuming? Greetings, Andres Freund -- Andres Freund

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
Great! If there's any further data I can supply to help, let me know. On Nov 18, 2013, at 2:15 PM, Andres Freund wrote: > Hi, > > Afaics it's likely a combination/interaction of bugs and fixes between: > * the initial HS code > * 5a031a5556ff83b8a9646892715d7fef415b83c3 > * f44eedc3f0f347a856e

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Andres Freund
Hi, Afaics it's likely a combination/interaction of bugs and fixes between: * the initial HS code * 5a031a5556ff83b8a9646892715d7fef415b83c3 * f44eedc3f0f347a856eea8590730769125964597 But that'd mean nobody noticed it during 9.3's beta... Greetings, Andres Freund -- Andres Freund

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 12:57 PM, Andres Freund wrote: > Were there any kind of patterns in the lost data? What kind of workload > are they running? I have an idea what the issue might be... On the P1 > S1 case, the data corrupted was data modified in the last few minutes before the switchover. I

Re: [HACKERS] unaccent module - two params function should be immutable

2013-11-18 Thread Bruce Momjian
On Fri, Nov 8, 2013 at 06:00:53PM -0500, Tom Lane wrote: > Bruce Momjian writes: > > [ mark unaccent functions immutable ] > > > Applied. > > This patch is flat out wrong and needs to be reverted. > > The functions were correctly marked (by you!) in commit > c0577c92a84cc477a88fe6868c16c4a7e33

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Andres Freund
On 2013-11-18 10:58:26 -0800, Christophe Pettus wrote: > After promotion, it was discovered that there was significant data > loss on S1. Rows that were present on P1 were missing on S1, and some > rows were duplicated (including duplicates that violated primary key > and other unique constraints)

Re: [HACKERS] Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-18 Thread Bruce Momjian
On Sat, Nov 9, 2013 at 02:15:52PM -0500, Robert Haas wrote: > On Fri, Nov 8, 2013 at 5:36 PM, Tom Lane wrote: > > [ I'm so far behind ... ] > > > > Bruce Momjian writes: > >> Applied. Thank you for all your suggestions. > > > > I thought the suggestion had been to issue a *warning*. How did th

Re: [HACKERS] GIN improvements part2: fast scan

2013-11-18 Thread Alexander Korotkov
On Fri, Nov 15, 2013 at 11:42 PM, Alexander Korotkov wrote: > On Fri, Nov 15, 2013 at 11:39 PM, Rod Taylor wrote: > >> >> >> >> On Fri, Nov 15, 2013 at 2:26 PM, Alexander Korotkov > > wrote: >> >>> On Fri, Nov 15, 2013 at 11:18 PM, Rod Taylor wrote: >>> 2%. It's essentially sentenc

Re: [HACKERS] Changing pg_dump default file format

2013-11-18 Thread Bruce Momjian
On Thu, Nov 7, 2013 at 02:40:17PM -0500, Peter Eisentraut wrote: > On 11/7/13, 1:00 PM, Josh Berkus wrote: > > If we wanted to change the defaults, I think it would be easier to > > create a separate bin name (e.g. pg_backup) than to change the existing > > parameters for pg_dump. > > Note the fo

Re: [HACKERS] CLUSTER FREEZE

2013-11-18 Thread Kevin Grittner
Josh Berkus wrote: > On 11/18/2013 08:39 AM, Bruce Momjian wrote: >> If we do add FREEZE, all we would be doing is delaying the time >> when all CLUSTER operations will use FREEZE, and hence debugging >> will be just as difficult.  My point is that will full >> knowledge, everyone would use FREEZE

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 12:00 PM, Christophe Pettus wrote: > One more correction: After rsync finished and the pg_base_backup() was > issued, the contents of pg_xlog/ on S1 were deleted. pg_stop_backup(), sorry. -- -- Christophe Pettus x...@thebuild.com -- Sent via pgsql-hackers mailing li

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 11:47 AM, Andres Freund wrote: > Without deleting any data, including pg_xlog/, backup.label, anything? One more correction: After rsync finished and the pg_base_backup() was issued, the contents of pg_xlog/ on S1 were deleted. -- -- Christophe Pettus x...@thebuild.com

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 11:47 AM, Andres Freund wrote: > Did you have hot_standby enabled on all of those machines? Even on the > 9.0.13 cluster? Actually, it's a bit more complex than this: 1. We don't know about P0, the 9.0.13 machine. I assume it was, but it was managed elsewhere. 2. P1 never

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 11:47 AM, Andres Freund wrote: > Without deleting any data, including pg_xlog/, backup.label, anything? Correct. > Did you have hot_standby enabled on all of those machines? Even on the > 9.0.13 cluster? Yes. > That was just recovery command and primary conninfo? Correct

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Andres Freund
On 2013-11-18 11:38:43 -0800, Christophe Pettus wrote: > > On Nov 18, 2013, at 11:28 AM, Andres Freund wrote: > > Could you detail how exactly the base backup was created? Including the > > *exact* logic for copying? > > 0. Before any of this began, P1 was archiving WAL segments to AWS-S3. > 1.

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 11:28 AM, Andres Freund wrote: > Could you detail how exactly the base backup was created? Including the > *exact* logic for copying? 0. Before any of this began, P1 was archiving WAL segments to AWS-S3. 1. pg_start_backup('', true) on P1. 2. Using rsync -av on P1, the entire

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Andres Freund
Hi, On 2013-11-18 10:58:26 -0800, Christophe Pettus wrote: > INCDIDENT #1: 9.0.14 -- A new secondary (S1) was initialized using > rsync off of an existing, correct primary (P1) for the base backup, > and using WAL-E for WAL segment shipping. Both the primary and > secondary were running 9.0.14.

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 10:58 AM, Christophe Pettus wrote: > As a note, P1 was created from another system (let's call it P0) using just > WAL shipping (no streaming replication), and no data corruption was observed. As another data point, P0 was running 9.0.13, rather than 9.0.14. -- -- Christophe

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Josh Berkus
On 11/18/2013 10:58 AM, Christophe Pettus wrote: > Three times in the last two weeks, we have experience data corruption > secondary servers using streaming replication on client systems. The > versions involved are 9.0.14, 9.2.5, and 9.3.1. Each incident was separate; > two cases they were fo

Re: [HACKERS] Improvement of pg_stat_statement usage about buffer hit ratio

2013-11-18 Thread Peter Geoghegan
On Mon, Nov 18, 2013 at 10:49 AM, Fujii Masao wrote: > The same idea was proposed before but not committed because > Itagaki thought that pg_stat_statements view should report only raw values. > Please read the following thread. I have the same feeling with him. > Anyway we should listen to more o

[HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
Three times in the last two weeks, we have experience data corruption secondary servers using streaming replication on client systems. The versions involved are 9.0.14, 9.2.5, and 9.3.1. Each incident was separate; two cases they were for the same client (9.0.14 and 9.3.1), one for a different

Re: [HACKERS] additional json functionality

2013-11-18 Thread Merlin Moncure
On Mon, Nov 18, 2013 at 12:10 PM, Josh Berkus wrote: > Merlin, > >> *) Aside from the text in and out routines, how is 'jsbonb' different >> from the coming 'nested hstore'? Enough to justify two code bases? > > In/out functions and defaults are all different. Otherwise, the two > types will be

Re: [HACKERS] Improvement of pg_stat_statement usage about buffer hit ratio

2013-11-18 Thread Fujii Masao
On Mon, Nov 18, 2013 at 8:36 PM, KONDO Mitsumasa wrote: > (2013/11/18 20:16), Haribabu kommi wrote: >> >> On 18 October 2013 13:35 KONDO Mitsumasa wrote: >>> >>> This patch conflicts pg_stat_statement_min_max_exectime patch which I >>> submitted, and pg_stat_statement_min_max_exectime patch also a

Re: [HACKERS] Review: pre-commit triggers

2013-11-18 Thread Dimitri Fontaine
Andrew Dunstan writes: > Given that, do we want to keep the bar on these operating in single user > mode? I can easily drop it and just document this way out of difficulty. Currently Event Triggers are disabled in single user mode, in parts because operating them require accessing to a catalog in

Re: [HACKERS] CLUSTER FREEZE

2013-11-18 Thread Josh Berkus
On 11/18/2013 08:39 AM, Bruce Momjian wrote: > If we do add FREEZE, all we would be doing is delaying the time when all > CLUSTER operations will use FREEZE, and hence debugging will be just as > difficult. My point is that will full knowledge, everyone would use > FREEZE unless they expect MVCC b

Re: [HACKERS] additional json functionality

2013-11-18 Thread Josh Berkus
Merlin, > *) Aside from the text in and out routines, how is 'jsbonb' different > from the coming 'nested hstore'? Enough to justify two code bases? In/out functions and defaults are all different. Otherwise, the two types will be accessing the same code base, so no duplication. Think of is a

Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-11-18 Thread Fujii Masao
On Tue, Nov 19, 2013 at 12:01 AM, Haribabu kommi wrote: > On 18 November 2013 18:45 Fujii Masao wrote: >> On Mon, Nov 18, 2013 at 6:31 PM, Haribabu kommi >> wrote: >> > >> > On 18 November 2013 11:19 Haribabu kommi wrote: >> >> On 17 November 2013 00:55 Fujii Masao wrote: >> >> > On Sat, Nov 16,

Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-11-18 Thread Andres Freund
On 2013-11-18 15:01:42 +, Haribabu kommi wrote: > > /* > + * Returns the malloced string of containing current working directory. > + * The caller has to take care of freeing the memory. > + * On failure exits with error code. > + */ > +static char * > +get_current_working_dir() > +{ > +

Re: [HACKERS] additional json functionality

2013-11-18 Thread Josh Berkus
On 11/18/2013 06:13 AM, Peter Eisentraut wrote: > On 11/15/13, 6:15 PM, Josh Berkus wrote: >> Thing is, I'm not particularly concerned about *Merlin's* specific use >> case, which there are ways around. What I am concerned about is that we >> may have users who have years of data stored in JSON tex

Re: [HACKERS] Turning recovery.conf into GUCs

2013-11-18 Thread Andres Freund
Hi, On 2013-11-15 22:38:05 -0500, Jaime Casanova wrote: > sorry, i clearly misunderstood you. attached a rebased patch with > those functions removed. * --write-standby-enable seems to loose quite some functionality in comparison to --write-recovery-conf since it doesn't seem to set primary_c

Re: [HACKERS] Review: pre-commit triggers

2013-11-18 Thread Andrew Dunstan
On 11/18/2013 09:39 AM, Ian Lawrence Barwick wrote: Review for Andrew Dunstan's patch in CF 2013-11: https://commitfest.postgresql.org/action/patch_view?id=1312 This review is more from a usage point of view, I would like to spend more time looking at the code but only so many hours in a da

Re: [HACKERS] CLUSTER FREEZE

2013-11-18 Thread Andres Freund
On 2013-11-18 11:39:44 -0500, Bruce Momjian wrote: > On Mon, Nov 18, 2013 at 09:22:58PM +1300, David Rowley wrote: > > So now I'm wondering what the next move should be for this patch? > > > > a. Are we waiting on Robert's patch to be committed before we can apply > > Thomas's > > cluster with fr

Re: [HACKERS] CLUSTER FREEZE

2013-11-18 Thread Bruce Momjian
On Mon, Nov 18, 2013 at 09:22:58PM +1300, David Rowley wrote: > So now I'm wondering what the next move should be for this patch? > > a. Are we waiting on Robert's patch to be committed before we can apply > Thomas's > cluster with freeze as default? > b. Are we waiting on me reviewing one or bot

Re: [HACKERS] appendPQExpBufferVA vs appendStringInfoVA

2013-11-18 Thread Heikki Linnakangas
On 18.11.2013 15:40, Marko Kreen wrote: On Mon, Nov 18, 2013 at 06:18:01PM +1300, David Rowley wrote: On Mon, Nov 18, 2013 at 1:01 AM, Marko Kreen wrote: I am bit suspicious of performance impact of this patch, but think that it's still worthwhile as it decreases code style where single string

Re: [HACKERS] nested hstore patch

2013-11-18 Thread Andres Freund
Hi, On 2013-11-12 22:35:31 +0400, Teodor Sigaev wrote: > Attatched patch adds nesting feature, types (string, boll and numeric > values), arrays and scalar to hstore type. I took a quick peek at this: * You cannot simply catch and ignore errors by doing + PG_TRY(); + { +

Re: [HACKERS] Freezing without write I/O

2013-11-18 Thread Andres Freund
On 2013-09-25 12:31:20 +0300, Heikki Linnakangas wrote: > Hmm, some of those are trivial, but others, rewrite_heap_tuple() are > currently only passed the HeapTuple, with no reference to the page where the > tuple came from. Instead of changing code to always pass that along with a > tuple, I think

Re: [HACKERS] Can we add sample systemd service file to git repo?

2013-11-18 Thread Will Crawford
On 12 November 2013 20:54, Nigel Heron wrote: > On Tue, Nov 12, 2013 at 11:47 AM, Devrim GÜNDÜZ wrote: ... >> http://svn.pgrpms.org/browser/rpm/redhat/9.3/postgresql/F-19/postgresql-9.3.service >> is an example of what we use in the RPMs. (if website fails, please just >> reload) >> >> Attached

Re: [HACKERS] Small improvement to json out functions by using cstring_to_text_with_len instead of cstring_to_text

2013-11-18 Thread Robert Haas
On Thu, Nov 14, 2013 at 2:18 AM, David Rowley wrote: > Hi, > > Here's a small patch which should speedup json out functions a little bit by > removing a call to strlen for which could be a long string. > The length of the string is already known by the StringInfoData, so there's > no point in lett

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-11-18 Thread Haribabu kommi
On 18 November 2013 20:01 Amit Kapila wrote: > On Mon, Nov 18, 2013 at 6:28 PM, Haribabu kommi > wrote: > > On 17 November 2013 12:25 Amit Kapila wrote: > >> On Sat, Nov 16, 2013 at 4:35 PM, Haribabu kommi > >> >> >> Find the rebased version attached with this mail. > >> >> > > >> >

Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-11-18 Thread Haribabu kommi
On 18 November 2013 18:45 Fujii Masao wrote: > On Mon, Nov 18, 2013 at 6:31 PM, Haribabu kommi > wrote: > > > > On 18 November 2013 11:19 Haribabu kommi wrote: > >> On 17 November 2013 00:55 Fujii Masao wrote: > >> > On Sat, Nov 16, 2013 at 2:27 PM, Haribabu kommi > >> > wrote: > >> > > on 15 Nov

Re: [HACKERS] inherit support for foreign tables

2013-11-18 Thread Tom Lane
Robert Haas writes: > On Thu, Nov 14, 2013 at 12:28 AM, Tom Lane wrote: >>> 2) Allow foreign tables to have CHECK constraints >>> Like NOT NULL, I think we don't need to enforce the check duroing >>> INSERT/UPDATE against foreign table. >> Really? > I think it's been previously proposed that we

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2013-11-18 Thread Heikki Linnakangas
On 14.10.2013 07:12, Peter Geoghegan wrote: On Wed, Oct 9, 2013 at 1:11 PM, Peter Geoghegan wrote: Unfortunately, I have a very busy schedule in the month ahead, including travelling to Ireland and Japan, so I don't think I'm going to get the opportunity to work on this too much. I'll try and p

Re: [HACKERS] additional json functionality

2013-11-18 Thread Andrew Dunstan
On 11/18/2013 09:38 AM, Merlin Moncure wrote: On Sun, Nov 17, 2013 at 10:19 PM, Andrew Dunstan wrote: I don't think any name that doesn't begin with "json" is acceptable. I could live with "jsonb". It has the merit of brevity, but maybe it's a tad too close to "json" to be the right answer. I

[HACKERS] Review: pre-commit triggers

2013-11-18 Thread Ian Lawrence Barwick
Review for Andrew Dunstan's patch in CF 2013-11: https://commitfest.postgresql.org/action/patch_view?id=1312 This review is more from a usage point of view, I would like to spend more time looking at the code but only so many hours in a day, etcetera; I hope this is useful as-is. Submission r

Re: [HACKERS] additional json functionality

2013-11-18 Thread Merlin Moncure
On Sun, Nov 17, 2013 at 10:19 PM, Andrew Dunstan wrote: > I don't think any name that doesn't begin with "json" is acceptable. I could > live with "jsonb". It has the merit of brevity, but maybe it's a tad too > close to "json" to be the right answer. I think that seems right. Couple thoughts:

Re: [HACKERS] inherit support for foreign tables

2013-11-18 Thread Robert Haas
On Thu, Nov 14, 2013 at 12:28 AM, Tom Lane wrote: >> 2) Allow foreign tables to have CHECK constraints >> Like NOT NULL, I think we don't need to enforce the check duroing >> INSERT/UPDATE against foreign table. > > Really? It's one thing to say that somebody who adds a CHECK constraint > to a fo

Re: [HACKERS] Force optimizer to use hash/nl/merge join?

2013-11-18 Thread Amit Kapila
On Mon, Nov 18, 2013 at 7:58 PM, Zhan Li wrote: > Hi All, > > Is there any way to force the optimizer to use a specific join operator? For > example, in SQL Server, I can do this way > > select * from (A inner hash join B on A.a = B.b) inner loop join C on A.a = > C.c > > I did some search but did

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-11-18 Thread Amit Kapila
On Mon, Nov 18, 2013 at 6:28 PM, Haribabu kommi wrote: > On 17 November 2013 12:25 Amit Kapila wrote: >> On Sat, Nov 16, 2013 at 4:35 PM, Haribabu kommi >> >> >> Find the rebased version attached with this mail. >> >> > >> > ereport(ERROR, >> > >> (errcode(ERRCODE_CONFIG_FI

[HACKERS] Force optimizer to use hash/nl/merge join?

2013-11-18 Thread Zhan Li
Hi All, Is there any way to force the optimizer to use a specific join operator? For example, in SQL Server, I can do this way select * from (A inner hash join B on A.a = B.b) inner loop join C on A.a = C.c I did some search but didn't find PostgreSQL had similar join hints except for enable_* s

Re: [HACKERS] additional json functionality

2013-11-18 Thread Peter Eisentraut
On 11/15/13, 6:15 PM, Josh Berkus wrote: > Thing is, I'm not particularly concerned about *Merlin's* specific use > case, which there are ways around. What I am concerned about is that we > may have users who have years of data stored in JSON text fields which > won't survive an upgrade to binary J

Re: [HACKERS] appendPQExpBufferVA vs appendStringInfoVA

2013-11-18 Thread Marko Kreen
On Mon, Nov 18, 2013 at 06:18:01PM +1300, David Rowley wrote: > On Mon, Nov 18, 2013 at 1:01 AM, Marko Kreen wrote: > > I am bit suspicious of performance impact of this patch, but think > > that it's still worthwhile as it decreases code style where single > > string argument is given to printf-s

Re: [HACKERS] CREATE TABLE IF NOT EXISTS AS

2013-11-18 Thread Fabrízio de Royes Mello
On Sun, Nov 17, 2013 at 6:05 PM, David E. Wheeler wrote: > On Nov 16, 2013, at 4:40 PM, Tom Lane wrote: > > >> Co-worker asked a question I could not answer: Why is IF NOT EXISTS not > supported by CREATE TABLE AS? > > > > That's an even worse idea than plain CREATE IF NOT EXISTS (which was > > p

Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-11-18 Thread Fujii Masao
On Mon, Nov 18, 2013 at 6:31 PM, Haribabu kommi wrote: > > On 18 November 2013 11:19 Haribabu kommi wrote: >> On 17 November 2013 00:55 Fujii Masao wrote: >> > On Sat, Nov 16, 2013 at 2:27 PM, Haribabu kommi >> > wrote: >> > > on 15 November 2013 17:26 Magnus Hagander wrote: >> > > >> > >>On Fri,

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-11-18 Thread Haribabu kommi
On 17 November 2013 12:25 Amit Kapila wrote: > On Sat, Nov 16, 2013 at 4:35 PM, Haribabu kommi > wrote: > > On 16 November 2013 09:43 Amit Kapila wrote: > >> On Fri, Nov 15, 2013 at 10:18 PM, Peter Eisentraut > >> wrote: > >> > On 11/14/13, 2:50 AM, Amit Kapila wrote: > >> >> Find the rebased ver

Re: [HACKERS] Sequence Access Method WIP

2013-11-18 Thread Heikki Linnakangas
On 14.11.2013 22:10, Simon Riggs wrote: Includes test extension which allows sequences without gaps - "gapless". I realize this is just for demonstration purposes, but it's worth noting that it doesn't actually guarantee that when you use the sequence to populate a column in the table, the co

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2013-11-18 Thread Andres Freund
On 2013-11-18 19:52:29 +0900, Michael Paquier wrote: > On Sat, Nov 16, 2013 at 5:09 AM, Andres Freund wrote: > > On 2013-11-15 11:40:17 +0900, Michael Paquier wrote: > >> - 20131114_3_reindex_concurrently.patch, providing the core feature. > >> Patch 3 needs to have patch 2 applied first. Regressi

Re: [HACKERS] Sequence Access Method WIP

2013-11-18 Thread Heikki Linnakangas
On 18.11.2013 13:48, Simon Riggs wrote: On 18 November 2013 07:50, Heikki Linnakangas wrote: It doesn't go far enough, it's still too *low*-level. The sequence AM implementation shouldn't need to have direct access to the buffer page at all. I don't think the sequence AM should be in contro

Re: [HACKERS] Sequence Access Method WIP

2013-11-18 Thread Simon Riggs
On 18 November 2013 07:50, Heikki Linnakangas wrote: > It doesn't go far enough, it's still too *low*-level. The sequence AM > implementation shouldn't need to have direct access to the buffer page at > all. > I don't think the sequence AM should be in control of 'cached'. The caching > is done

Re: [HACKERS] Sequence Access Method WIP

2013-11-18 Thread Andres Freund
On 2013-11-18 12:50:21 +0200, Heikki Linnakangas wrote: > On 18.11.2013 11:48, Andres Freund wrote: > I don't think the sequence AM should be in control of 'cached'. The caching > is done outside the AM. And log_cnt probably should be passed to the _alloc > function directly as an argument, ie. the

  1   2   >