[HACKERS] Scaling shared buffer eviction

2014-05-14 Thread Amit Kapila
As mentioned previously about my interest in improving shared buffer eviction especially by reducing contention around BufFreelistLock, I would like to share my progress about the same. The test used for this work is mainly the case when all the data doesn't fit in shared buffers, but does fit in

Re: [HACKERS] Freezing without write I/O

2014-05-14 Thread David Fetter
On Wed, May 14, 2014 at 05:46:49PM -0700, Jeff Janes wrote: > On Monday, January 27, 2014, Robert Haas wrote: > > > On Mon, Jan 27, 2014 at 4:16 PM, Simon Riggs > > > > > wrote: > > > On 26 January 2014 12:58, Andres Freund > > > > > > wrote: > > >> On 2014-01-25 20:26:16 -0800, Peter Geoghegan

Re: [HACKERS] Freezing without write I/O

2014-05-14 Thread Jeff Janes
On Monday, January 27, 2014, Robert Haas wrote: > On Mon, Jan 27, 2014 at 4:16 PM, Simon Riggs > > > wrote: > > On 26 January 2014 12:58, Andres Freund > > > > wrote: > >> On 2014-01-25 20:26:16 -0800, Peter Geoghegan wrote: > >>> Shouldn't this patch be in the January commitfest? > >> > >> I t

Re: [HACKERS] [PATCH] Writing changes of decoding plugin in the memory context where data is kept?

2014-05-14 Thread Michael Paquier
On Thu, May 15, 2014 at 9:18 AM, Andres Freund wrote: > On 2014-05-15 09:11:15 +0900, Michael Paquier wrote: >> When working on a decoder plugin, I have been pointed that it is >> incorrect to write changes in an output plugin while not being in the >> memory context where changes are written. > >

Re: [HACKERS] buildfarm / handling (undefined) locales

2014-05-14 Thread Tom Lane
Tomas Vondra writes: > On 13.5.2014 23:07, Tom Lane wrote: >> Attached is a draft patch that behaves like this: >> >> $ initdb --locale=foo >> The files belonging to this database system will be owned by user "postgres". >> This user must also own the server process. >> >> initdb: invalid locale

Re: [HACKERS] buildfarm / handling (undefined) locales

2014-05-14 Thread Tomas Vondra
On 13.5.2014 23:07, Tom Lane wrote: > Andrew Dunstan writes: >> On 05/13/2014 04:14 PM, Tom Lane wrote: >>> But independently of whether it's a fatal error or not: when there's >>> no relevant command-line argument then we print the >>> >>> invalid locale name "" >>> >>> message which is surely pr

Re: [HACKERS] [PATCH] Writing changes of decoding plugin in the memory context where data is kept?

2014-05-14 Thread Andres Freund
On 2014-05-15 09:11:15 +0900, Michael Paquier wrote: > When working on a decoder plugin, I have been pointed that it is > incorrect to write changes in an output plugin while not being in the > memory context where changes are written. That was a misunderstanding. It's perfectly alright to do so.

Re: [HACKERS] Cache invalidation bug in RelationGetIndexAttrBitmap()

2014-05-14 Thread Tom Lane
Tomas Vondra writes: > On 14.5.2014 22:29, Andres Freund wrote: >> I'm afraid it's more in the year range from what i've seen. I.e. not >> practical. > Yeah, that wouldn't be very practical. I'll try to run it though and if > it'd run more than a few days, I'll switch it to CLOBBER_CACHE_ALWAYS.

[HACKERS] [PATCH] Writing changes of decoding plugin in the memory context where data is kept?

2014-05-14 Thread Michael Paquier
Hi all, When working on a decoder plugin, I have been pointed that it is incorrect to write changes in an output plugin while not being in the memory context where changes are written. In pg_decode_change@test_decoding.c, we do the following: old = MemoryContextSwitchTo(data->context); OutputPlugi

Re: [HACKERS] Sending out a request for more buildfarm animals?

2014-05-14 Thread Tomas Vondra
On 13.5.2014 20:42, Tomas Vondra wrote: > On 10.5.2014 20:21, Tomas Vondra wrote: >> On 9.5.2014 00:47, Tomas Vondra wrote: >> >> And I've requested 6 more animals - two for each compiler. One set for >> tests with basic CLOBBER, one set for recursive CLOBBER. > > Can someone please approve the an

Re: [HACKERS] Cache invalidation bug in RelationGetIndexAttrBitmap()

2014-05-14 Thread Tomas Vondra
On 14.5.2014 22:29, Andres Freund wrote: > Hi, > > On 2014-05-14 21:04:41 +0200, Tomas Vondra wrote: >> On 14.5.2014 17:52, Andres Freund wrote: >>> On 2014-05-14 15:17:39 +0200, Andres Freund wrote: On 2014-05-14 15:08:08 +0200, Tomas Vondra wrote: > Apparently there's something wrong wi

Re: [HACKERS] Cache invalidation bug in RelationGetIndexAttrBitmap()

2014-05-14 Thread Andres Freund
Hi, On 2014-05-14 21:04:41 +0200, Tomas Vondra wrote: > On 14.5.2014 17:52, Andres Freund wrote: > > On 2014-05-14 15:17:39 +0200, Andres Freund wrote: > >> On 2014-05-14 15:08:08 +0200, Tomas Vondra wrote: > >>> Apparently there's something wrong with 'test-decoding-check': > >> > >> Man. I shoul

Re: [HACKERS] Cache invalidation bug in RelationGetIndexAttrBitmap()

2014-05-14 Thread Tomas Vondra
On 14.5.2014 17:52, Andres Freund wrote: > On 2014-05-14 15:17:39 +0200, Andres Freund wrote: >> On 2014-05-14 15:08:08 +0200, Tomas Vondra wrote: >>> Apparently there's something wrong with 'test-decoding-check': >> >> Man. I shouldn't have asked... My code. There's some output in there >> that's

Re: [HACKERS] 9.4 release notes

2014-05-14 Thread Bruce Momjian
On Wed, May 14, 2014 at 05:40:04PM +0100, Dean Rasheed wrote: > > Agreed. Adjusted attached patch applied. I was never happy with the > > previous awkward auto-update wording. > > > > Thanks. There's a typo in the new text though --- "cals" should be "calls". Thanks, fixed. -- Bruce Momjian

Re: [HACKERS] Cache invalidation bug in RelationGetIndexAttrBitmap()

2014-05-14 Thread Tom Lane
Andres Freund writes: > On 2014-05-14 12:15:27 -0400, Tom Lane wrote: >> And why does the header >> comment for RelationGetIndexList make no mention of this new side-effect? >> Somebody did a seriously poor job of adding this functionality to >> relcache. > It's not like it's not documented: Ther

[HACKERS] Various cosmetic fixes

2014-05-14 Thread Euler Taveira
Hi, While updating pt-br translation I noticed that some sentences could be improved. I also fix some style glitches. A set of patches are attached. -- Euler Taveira Timbira - http://www.timbira.com.br/ PostgreSQL: Consultoria, Desenvolvimento, Suporte 24x7 e Treinamento

Re: [HACKERS] Cache invalidation bug in RelationGetIndexAttrBitmap()

2014-05-14 Thread Andres Freund
On 2014-05-14 13:32:43 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-05-14 12:15:27 -0400, Tom Lane wrote: > >> And why does the header > >> comment for RelationGetIndexList make no mention of this new side-effect? > >> Somebody did a seriously poor job of adding this functionality to

Re: [HACKERS] 9.4 release notes

2014-05-14 Thread Dean Rasheed
On 14 May 2014 15:07, Bruce Momjian wrote: > On Wed, May 14, 2014 at 08:33:15AM +0100, Dean Rasheed wrote: >> On 4 May 2014 13:46, Bruce Momjian wrote: >> > I have completed the initial version of the 9.4 release notes. You can >> > view them here: >> > >> > http://www.postgresql.org/doc

Re: [HACKERS] Typo in release notes

2014-05-14 Thread Bruce Momjian
fOn Wed, May 14, 2014 at 07:40:03AM +0400, Sergey Muraviov wrote: > Hi. > > Here's a patch that fixes it. Thanks, applied. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. + -- Sent via pgsql-hac

Re: [HACKERS] Cache invalidation bug in RelationGetIndexAttrBitmap()

2014-05-14 Thread Andres Freund
On 2014-05-14 12:15:27 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-05-14 15:17:39 +0200, Andres Freund wrote: > >> My gut feeling says it's in RelationGetIndexList(). > > > Nearly right. It's in RelationGetIndexAttrBitmap(). Fix attached. > > TBH, I don't believe this patch at all

Re: [HACKERS] btree_gist valgrind warnings about uninitialized memory

2014-05-14 Thread Tom Lane
Andres Freund writes: > On 2014-05-14 10:07:18 -0400, Tom Lane wrote: >> I think that's an OK restriction as long as we warn people about it >> (you could update a replication pair as long as you shut them both >> down cleanly at the same time, right?). Can the WAL replay routine >> be made to de

Re: [HACKERS] Cache invalidation bug in RelationGetIndexAttrBitmap()

2014-05-14 Thread Tom Lane
Andres Freund writes: > On 2014-05-14 15:17:39 +0200, Andres Freund wrote: >> My gut feeling says it's in RelationGetIndexList(). > Nearly right. It's in RelationGetIndexAttrBitmap(). Fix attached. TBH, I don't believe this patch at all. Where exactly is rd_replidindex reset? If it's supposed

Re: [HACKERS] btree_gist valgrind warnings about uninitialized memory

2014-05-14 Thread Andres Freund
On 2014-05-14 10:07:18 -0400, Tom Lane wrote: > Heikki Linnakangas writes: > > On 05/13/2014 05:13 PM, Andres Freund wrote: > >> What's your plans with your spgist fix? Commit it once 9.5 is branched? > > > Good question. I don't know. I would still like to commit it to 9.4. It > > doesn't requi

Re: [HACKERS] Wanna help PostgreSQL

2014-05-14 Thread Merlin Moncure
On Wed, May 14, 2014 at 10:58 AM, Tom Lane wrote: > Merlin Moncure writes: >> On Wed, May 14, 2014 at 9:44 AM, Tom Lane wrote: >>> The reason that that project has gone untouched for upwards of ten years >>> is that it's not just a large coding project, but involves a lot of >>> complex API desi

Re: [HACKERS] Wanna help PostgreSQL

2014-05-14 Thread Tom Lane
Merlin Moncure writes: > On Wed, May 14, 2014 at 9:44 AM, Tom Lane wrote: >> The reason that that project has gone untouched for upwards of ten years >> is that it's not just a large coding project, but involves a lot of >> complex API design with uncertain goals. It's not very clear what >> fea

Re: [HACKERS] Wanna help PostgreSQL

2014-05-14 Thread Andres Freund
On 2014-05-14 10:49:05 -0500, Merlin Moncure wrote: > On Wed, May 14, 2014 at 9:44 AM, Tom Lane wrote: > > The reason that that project has gone untouched for upwards of ten years > > is that it's not just a large coding project, but involves a lot of > > complex API design with uncertain goals.

[HACKERS] Cache invalidation bug in RelationGetIndexAttrBitmap()

2014-05-14 Thread Andres Freund
On 2014-05-14 15:17:39 +0200, Andres Freund wrote: > On 2014-05-14 15:08:08 +0200, Tomas Vondra wrote: > > Apparently there's something wrong with 'test-decoding-check': > > Man. I shouldn't have asked... My code. There's some output in there > that's probably triggered by the extraordinarily long

Re: [HACKERS] Wanna help PostgreSQL

2014-05-14 Thread Merlin Moncure
On Wed, May 14, 2014 at 9:44 AM, Tom Lane wrote: > The reason that that project has gone untouched for upwards of ten years > is that it's not just a large coding project, but involves a lot of > complex API design with uncertain goals. It's not very clear what > features people would want from a

Re: [HACKERS] 9.5: UPDATE/DELETE .. ORDER BY .. LIMIT ..

2014-05-14 Thread Robert Haas
On Sun, May 11, 2014 at 12:47 PM, Tom Lane wrote: > Simon Riggs writes: >> On 11 May 2014 11:18, Andres Freund wrote: >>> I don't know. I'd find UPDATE/DELETE ORDER BY something rather >>> useful. > >> Perhaps if an index exists to provide an ordering that makes it clear >> what this means, then

Re: [HACKERS] buildfarm / handling (undefined) locales

2014-05-14 Thread Andrew Dunstan
On 05/14/2014 11:02 AM, Tom Lane wrote: I wrote: As far as Windows goes, we could certainly use #ifdef WIN32 to print some different text, if anyone can write down what it should be. Some googling suggests that on Windows, setlocale pays no attention to environment variables but gets a value t

[HACKERS] Re: popen and pclose redefinitions causing many warning in Windows build

2014-05-14 Thread Noah Misch
On Wed, May 14, 2014 at 05:51:24PM +0300, Heikki Linnakangas wrote: > On 05/14/2014 05:37 PM, Noah Misch wrote: > >On Wed, May 14, 2014 at 03:15:38PM +0300, Heikki Linnakangas wrote: > >>On 05/09/2014 02:56 AM, Noah Misch wrote: > >>>MinGW: > >>>http://sourceforge.net/p/mingw/mingw-org-wsl/ci/mast

Re: [HACKERS] buildfarm / handling (undefined) locales

2014-05-14 Thread Tom Lane
I wrote: > As far as Windows goes, we could certainly use #ifdef WIN32 to print > some different text, if anyone can write down what it should be. Some googling suggests that on Windows, setlocale pays no attention to environment variables but gets a value that the user can set via the Control Pan

[HACKERS] Re: popen and pclose redefinitions causing many warning in Windows build

2014-05-14 Thread Heikki Linnakangas
On 05/14/2014 05:37 PM, Noah Misch wrote: On Wed, May 14, 2014 at 03:15:38PM +0300, Heikki Linnakangas wrote: On 05/09/2014 02:56 AM, Noah Misch wrote: MinGW: http://sourceforge.net/p/mingw/mingw-org-wsl/ci/master/tree/include/stdio.h#l467 MinGW-w64: http://sourceforge.net/p/mingw-w64/code/HE

Re: [HACKERS] Wanna help PostgreSQL

2014-05-14 Thread Tom Lane
Zhe-Wei Jiang writes: >> I read the GSoC ideas page and think the only one I'm possibly to help for >> now is >> "Rewrite (add) pg_dump and pg_restore utilities as libraries (.so, .dll & >> .dylib)", >> while I'm still not clear with the whole postgreSQL project. >> >> If my understanding is corr

[HACKERS] Re: popen and pclose redefinitions causing many warning in Windows build

2014-05-14 Thread Noah Misch
On Wed, May 14, 2014 at 03:15:38PM +0300, Heikki Linnakangas wrote: > On 05/09/2014 02:56 AM, Noah Misch wrote: > >MinGW: > >http://sourceforge.net/p/mingw/mingw-org-wsl/ci/master/tree/include/stdio.h#l467 > >MinGW-w64: > >http://sourceforge.net/p/mingw-w64/code/HEAD/tree/trunk/mingw-w64-headers/

Re: [HACKERS] Wanna help PostgreSQL

2014-05-14 Thread Zhe-Wei Jiang
Thanks Jeff. 2014-05-14 2:19 GMT+08:00 Jeff Janes : > On Tue, May 13, 2014 at 10:12 AM, Zhe-Wei Jiang wrote: > >> Hi everyone, >> >> I'm new to the open source community, and wanna help in some >> implementation work. >> However, when I looked into the TODO list, I found most of the items are >>

Re: [HACKERS] Wanna help PostgreSQL

2014-05-14 Thread Zhe-Wei Jiang
See, that's beyond my imagination. Then I have to do more research first before I can understand the other ideas. Regards, Zhe-Wei Jiang 2014-05-14 22:25 GMT+08:00 Stephen Frost : > Zhe-Wei, > > * Zhe-Wei Jiang (jrreinha...@gmail.com) wrote: > > I read the GSoC ideas page and think the only on

Re: [HACKERS] buildfarm / handling (undefined) locales

2014-05-14 Thread Tomas Vondra
On 14 Květen 2014, 16:26, Heikki Linnakangas wrote: > On 05/14/2014 05:19 PM, Tom Lane wrote: > >> Anyway, given that we seem to have consensus on doing this (modulo >> exact text of the error message), the next question is whether we >> want to change this only in HEAD, or consider it a back-patch

Re: [HACKERS] buildfarm / handling (undefined) locales

2014-05-14 Thread Heikki Linnakangas
On 05/14/2014 05:19 PM, Tom Lane wrote: Heikki Linnakangas writes: On 05/14/2014 12:07 AM, Tom Lane wrote: Well, all we know is that we asked setlocale for the default locale from the environment and it failed. We don't really know which variable(s) it looked at. Printing the values of the

[HACKERS] Re: popen and pclose redefinitions causing many warning in Windows build

2014-05-14 Thread Andrew Dunstan
On 05/14/2014 08:15 AM, Heikki Linnakangas wrote: On 05/09/2014 02:56 AM, Noah Misch wrote: On Thu, May 08, 2014 at 12:14:44PM -0400, Tom Lane wrote: Andrew Dunstan writes: I'm pretty sure we need this on Mingw - this SYSTEMQUOTE stuff dates back well before 8.3, IIRC, which is when we first

Re: [HACKERS] buildfarm / handling (undefined) locales

2014-05-14 Thread Tom Lane
Heikki Linnakangas writes: > On 05/14/2014 12:07 AM, Tom Lane wrote: >> Well, all we know is that we asked setlocale for the default locale from >> the environment and it failed. We don't really know which variable(s) >> it looked at. > Printing the values of the environment variables sounds com

Re: [HACKERS] Wanna help PostgreSQL

2014-05-14 Thread Zhe-Wei Jiang
Forgot to include pgsql-hackers. 2014-05-14 22:11 GMT+08:00 Zhe-Wei Jiang : > Thanks Stephen. > > I read the GSoC ideas page and think the only one I'm possibly to help for > now is > "Rewrite (add) pg_dump and pg_restore utilities as libraries (.so, .dll & > .dylib)", > while I'm still not clear

Re: [HACKERS] btree_gist valgrind warnings about uninitialized memory

2014-05-14 Thread Tom Lane
Heikki Linnakangas writes: > On 05/13/2014 05:13 PM, Andres Freund wrote: >> What's your plans with your spgist fix? Commit it once 9.5 is branched? > Good question. I don't know. I would still like to commit it to 9.4. It > doesn't require catalog changes, but it's an incompatible change in the

Re: [HACKERS] 9.4 release notes

2014-05-14 Thread Bruce Momjian
On Wed, May 14, 2014 at 08:33:15AM +0100, Dean Rasheed wrote: > On 4 May 2014 13:46, Bruce Momjian wrote: > > I have completed the initial version of the 9.4 release notes. You can > > view them here: > > > > http://www.postgresql.org/docs/devel/static/release-9-4.html > > > > I will be a

Re: [HACKERS] Sending out a request for more buildfarm animals?

2014-05-14 Thread Andres Freund
On 2014-05-14 15:08:08 +0200, Tomas Vondra wrote: > On 14 Květen 2014, 13:51, Andres Freund wrote: > > On 2014-05-13 20:42:16 +0200, Tomas Vondra wrote: > >> Can someone please approve the animals I've requested a few days ago? > >> I'm already running the clobber tests with '--nosend --nostatus' a

Re: [HACKERS] Sending out a request for more buildfarm animals?

2014-05-14 Thread Tomas Vondra
On 14 Květen 2014, 13:51, Andres Freund wrote: > On 2014-05-13 20:42:16 +0200, Tomas Vondra wrote: >> Can someone please approve the animals I've requested a few days ago? >> I'm already running the clobber tests with '--nosend --nostatus' and >> it's already reporting some errors. Would be nice to

Re: [HACKERS] btree_gist valgrind warnings about uninitialized memory

2014-05-14 Thread Andres Freund
On 2014-05-14 14:55:38 +0300, Heikki Linnakangas wrote: > On 05/13/2014 05:13 PM, Andres Freund wrote: > >That leaves the spgist thing and some infrastructure till we can setup a > >valgrind animal... From what we've caught with it so far that seems > >rather worthwile. > >What's your plans with yo

[HACKERS] Re: popen and pclose redefinitions causing many warning in Windows build

2014-05-14 Thread Heikki Linnakangas
On 05/09/2014 02:56 AM, Noah Misch wrote: On Thu, May 08, 2014 at 12:14:44PM -0400, Tom Lane wrote: Andrew Dunstan writes: I'm pretty sure we need this on Mingw - this SYSTEMQUOTE stuff dates back well before 8.3, IIRC, which is when we first got full MSVC support. I tried googling for some

Re: [HACKERS] btree_gist valgrind warnings about uninitialized memory

2014-05-14 Thread Heikki Linnakangas
On 05/13/2014 05:13 PM, Andres Freund wrote: That leaves the spgist thing and some infrastructure till we can setup a valgrind animal... From what we've caught with it so far that seems rather worthwile. What's your plans with your spgist fix? Commit it once 9.5 is branched? Good question. I do

Re: [HACKERS] Sending out a request for more buildfarm animals?

2014-05-14 Thread Andres Freund
On 2014-05-13 20:42:16 +0200, Tomas Vondra wrote: > Can someone please approve the animals I've requested a few days ago? > I'm already running the clobber tests with '--nosend --nostatus' and > it's already reporting some errors. Would be nice to get it to the > buildfarm. Can you provide some de

Re: [HACKERS] buildfarm / handling (undefined) locales

2014-05-14 Thread Heikki Linnakangas
On 05/14/2014 12:07 AM, Tom Lane wrote: Andrew Dunstan writes: On 05/13/2014 04:14 PM, Tom Lane wrote: But independently of whether it's a fatal error or not: when there's no relevant command-line argument then we print the invalid locale name "" message which is surely pretty unhelpful. It

Re: [HACKERS] gettimeofday is at the end of its usefulness?

2014-05-14 Thread Robert Haas
On Tue, May 13, 2014 at 11:34 PM, Greg Stark wrote: > I always assumed the kernel used rdtsc to implement some of the high > performance timers. It can save the current time in a mapped page when > it schedules a process and then in the vdso syscall (ie in user-space) > it can use rdtsc to calcula

Re: [HACKERS] gettimeofday is at the end of its usefulness?

2014-05-14 Thread Ants Aasma
On Wed, May 14, 2014 at 6:34 AM, Greg Stark wrote: > I always assumed the kernel used rdtsc to implement some of the high > performance timers. It can save the current time in a mapped page when > it schedules a process and then in the vdso syscall (ie in user-space) > it can use rdtsc to calculat

Re: [HACKERS] Error in running DBT2

2014-05-14 Thread Rohit Goyal
Hi Peter, I tried the solution suggested by you. Please problem still persists. I run the the *dbt2-pgsql-build-db -w 1 * but, after some time, I faced this error */home/abhi/dbt2_install/bin/dbt2-pgsql-load-stored-procs: 45: [: c: unexpected operator* */home/abhi/dbt2_install/bin/dbt2-pgsql-lo

Re: [HACKERS] 9.4 release notes

2014-05-14 Thread Dean Rasheed
On 4 May 2014 13:46, Bruce Momjian wrote: > I have completed the initial version of the 9.4 release notes. You can > view them here: > > http://www.postgresql.org/docs/devel/static/release-9-4.html > > I will be adding additional markup in the next few days. > > Feedback expected and welc