Re: [HACKERS] JSON for PG 9.2

2012-01-10 Thread Pavel Stehule
2012/1/11 Robert Haas : > On Tue, Dec 20, 2011 at 9:06 PM, David E. Wheeler > wrote: >> On Dec 20, 2011, at 10:39 AM, Claes Jakobsson wrote: >>> Are people explicitly asking for a) *JSON* datatype or b) a type that lets >>> you store arbitrary complex semi-untyped data structures? >> >> Yes. >>

Re: [HACKERS] checkpoint writeback via sync_file_range

2012-01-10 Thread Greg Smith
On 1/10/12 9:14 PM, Robert Haas wrote: Based on that, I whipped up the attached patch, which, if sync_file_range is available, simply iterates through everything that will eventually be fsync'd before beginning the write phase and tells the Linux kernel to put them all under write-out. I hadn't

Re: [HACKERS] Sending notifications from the master to the standby

2012-01-10 Thread Tom Lane
BTW ... it occurs to me to ask whether we really have a solid use-case for having listeners attached to slave servers. I have personally never seen an application for LISTEN/NOTIFY in which the listeners were entirely read-only. Even if there are one or two cases out there, it's not clear to me t

Re: [HACKERS] Sending notifications from the master to the standby

2012-01-10 Thread Tom Lane
Joachim Wieland writes: > On Tue, Jan 10, 2012 at 11:55 AM, Tom Lane wrote: >> Simon Riggs writes: >>> Suggest we add something to initial handshake from standby to say >>> "please send me notify traffic", >> +1 on that. > From what you said I imagined this walsender listener as a regular > li

Re: [HACKERS] Real-life range datasets

2012-01-10 Thread Stefan Keller
Hi I'm proposing OpenStreetMap which is of variable size up to >250 GB XML Data for whole world. It's downloadable from CloudMade.com or Geofabrik.de and can be imported into PostgreSQL using osm2pgsql. It's a key/value schema literally of the real world. I'm using hstore option of osm2pgsql and h

[HACKERS] checkpoint writeback via sync_file_range

2012-01-10 Thread Robert Haas
Greg Smith muttered a while ago about wanting to do something with sync_file_range to improve checkpoint behavior on Linux. I thought he was talking about trying to sync only the range of blocks known to be dirty, which didn't seem like a very exciting idea, but after looking at the man page for s

Re: [HACKERS] JSON for PG 9.2

2012-01-10 Thread Robert Haas
On Tue, Dec 20, 2011 at 9:06 PM, David E. Wheeler wrote: > On Dec 20, 2011, at 10:39 AM, Claes Jakobsson wrote: >> Are people explicitly asking for a) *JSON* datatype or b) a type that lets >> you store arbitrary complex semi-untyped data structures? > > Yes. > >> if b) then this might get a lot

Re: [HACKERS] log messages for archive recovery progress

2012-01-10 Thread Euler Taveira de Oliveira
On 08-01-2012 11:59, Satoshi Nagayasu / Uptime Technologies, LLC. wrote: >> [2011-12-08 15:14:36 JST] 16758: LOG: restored log file >> "00080046" from archive >> [2011-12-08 15:14:36 JST] 16758: LOG: recoverying 00080046 >> [2011-12-08 15:14:36 JST] 16758: LOG: r

Re: [HACKERS] Sending notifications from the master to the standby

2012-01-10 Thread Joachim Wieland
On Tue, Jan 10, 2012 at 11:55 AM, Tom Lane wrote: > Simon Riggs writes: > [ Tom sketches a design ] > Seems a bit overcomplicated.  I was just thinking of having walreceiver > note the WAL endpoint at the instant of receipt of a notify message, > and not release the notify message to the slave ri

Re: [HACKERS] pgsphere

2012-01-10 Thread Jan-Benedict Glaw
On Wed, 2012-01-11 00:43:25 +0400, Oleg Bartunov wrote: > On Tue, 10 Jan 2012, Jan-Benedict Glaw wrote: > > On Sun, 2012-01-08 22:19:53 +0400, Oleg Bartunov wrote: > > > pgsphere now could benefit very much from our KNNGiST feature. > > > This is number one development from my point of view. I an

Re: [HACKERS] pgsphere

2012-01-10 Thread Greg Smith
On 1/10/12 9:54 AM, Andrew Dunstan wrote: Speaking with my pgfoundry admin hat on, I am extremely reluctant to take control of a project away from its owners. If you don't get any action in a week or so, you can approach the pgfoundry admins for help. In the meantime you might want to fork the co

Re: [HACKERS] Command Triggers

2012-01-10 Thread Dimitri Fontaine
Hi, Robert Haas writes: > Maybe we should try to split the baby here and defer the question of > whether to expose any of the parse tree internals, and if so how much, > to a future release. It seems to me that we could design a fairly > useful set of functionality around AFTER-CREATE, BEFORE-DR

Re: [HACKERS] pgsphere

2012-01-10 Thread Oleg Bartunov
On Tue, 10 Jan 2012, Jan-Benedict Glaw wrote: On Sun, 2012-01-08 22:19:53 +0400, Oleg Bartunov wrote: pgsphere now could benefit very much from our KNNGiST feature. This is number one development from my point of view. I and Teodor have no time to work on pgsphere, sorry. But, there are some a

Re: [HACKERS] random_page_cost vs seq_page_cost

2012-01-10 Thread Greg Smith
On 1/5/12 5:04 AM, Benedikt Grundmann wrote: I have a question of how to benchmark hardware to determine the appropriate ratio of seq_page_cost vs random_page_cost. Emails in this mailing lists archive seem to indicate that 1.0 vs 3.0 - 4.0 are appropriate values on modern hardware. Which surpr

Re: [HACKERS] libpq: PQcmdStatus, PQcmdTuples signatures can be painlessly improved

2012-01-10 Thread Alex Goncharov
,--- You/Peter (Tue, 10 Jan 2012 19:13:42 +0200) * | On tis, 2011-12-13 at 07:55 -0500, Alex Goncharov wrote: | > char *PQcmdStatus(PGresult *res); | > char *PQcmdTuples(PGresult *res); | > | > Unreasonable: | > | > a. What, these two can modify 'res' I pass in?.. | > | > b. Oh, yes,

Re: [HACKERS] Add SPI results constants available for PL/*

2012-01-10 Thread Pavel Stehule
2012/1/10 Andrew Dunstan : > > > On 01/10/2012 12:34 PM, Pavel Stehule wrote: >>> >>> >>> Actually, now I look closer I see that PLPerl passes back a stringified >>> status from SPI_execute(), so there is no great need for setting up these >>> constants. It's probably water under the bridge now, bu

Re: [HACKERS] Add SPI results constants available for PL/*

2012-01-10 Thread Andrew Dunstan
On 01/10/2012 12:34 PM, Pavel Stehule wrote: Actually, now I look closer I see that PLPerl passes back a stringified status from SPI_execute(), so there is no great need for setting up these constants. It's probably water under the bridge now, but maybe PLPython should have done this too. Th

Re: [HACKERS] pgsphere

2012-01-10 Thread Jan-Benedict Glaw
On Sun, 2012-01-08 22:19:53 +0400, Oleg Bartunov wrote: > pgsphere now could benefit very much from our KNNGiST feature. This is > number one development from my point of view. I and Teodor have no > time to work on pgsphere, sorry. But, there are some astronomers I'm working > with, who can take

Re: [HACKERS] Sending notifications from the master to the standby

2012-01-10 Thread Simon Riggs
On Tue, Jan 10, 2012 at 4:55 PM, Tom Lane wrote: > Simon Riggs writes: >> On Tue, Jan 10, 2012 at 5:00 AM, Tom Lane wrote: >>> It might be a bit tricky to get walreceivers to inject >>> the data into the slave-side ring buffer at the right time, ie, not >>> until after the commit a given message

Re: [HACKERS] Add SPI results constants available for PL/*

2012-01-10 Thread Pavel Stehule
2012/1/10 Andrew Dunstan : > > > On 01/03/2012 09:11 PM, Andrew Dunstan wrote: >> >> >> >> On 01/03/2012 08:40 PM, Robert Haas wrote: >>> >>> On Tue, Jan 3, 2012 at 9:33 AM, Pavel Stehule >>>  wrote: > > I'd suppose it interesting to add a table to pg_catalog containing this > data. >>>

Re: [HACKERS] Add SPI results constants available for PL/*

2012-01-10 Thread Andrew Dunstan
On 01/03/2012 09:11 PM, Andrew Dunstan wrote: On 01/03/2012 08:40 PM, Robert Haas wrote: On Tue, Jan 3, 2012 at 9:33 AM, Pavel Stehule wrote: I'd suppose it interesting to add a table to pg_catalog containing this data. - it is useless overhead I tend to agree. I am expecting so defini

Re: [HACKERS] libpq: PQcmdStatus, PQcmdTuples signatures can be painlessly improved

2012-01-10 Thread Peter Eisentraut
On tis, 2011-12-13 at 07:55 -0500, Alex Goncharov wrote: > char *PQcmdStatus(PGresult *res); > char *PQcmdTuples(PGresult *res); > > Unreasonable: > > a. What, these two can modify 'res' I pass in?.. > > b. Oh, yes, because they return 'char *' pointing to > 'res->cmdStatus+n', so,

Re: [HACKERS] Sending notifications from the master to the standby

2012-01-10 Thread Tom Lane
Simon Riggs writes: > On Tue, Jan 10, 2012 at 5:00 AM, Tom Lane wrote: >> It might be a bit tricky to get walreceivers to inject >> the data into the slave-side ring buffer at the right time, ie, not >> until after the commit a given message describes has been replayed; >> but I don't immediately

Re: [HACKERS] Sending notifications from the master to the standby

2012-01-10 Thread Simon Riggs
On Tue, Jan 10, 2012 at 12:56 PM, Joachim Wieland wrote: > I chose to do it this way because it seemed like the most natural way > to do it (which of course doesn't mean it's the best)  :-). If its any consolation its exactly how I would have done it also up until about 2 months ago, and I remem

Re: [HACKERS] pgsphere

2012-01-10 Thread Oleg Bartunov
I think Dave you fork project, I suggest different name pgsphere-2 to avoid confusion. github would be ok. Teodor and I are really busy guys, so I don't believe we could participate much, except discussion and testing. We implemented KNNGiST, you add neighbourhood search support to pgsphere :)

Re: [HACKERS] pgsphere

2012-01-10 Thread Andrew Dunstan
On 01/10/2012 09:04 AM, Dave Cramer wrote: So where do we go from here ? First, please note that -hackers is not the right place for this discussion. pgsphere is a pgfoundry project, which is not the province of -hackers. As I suggested, the best solution is for Teodor to add you as a p

Re: [HACKERS] LWLOCK_STATS

2012-01-10 Thread Merlin Moncure
On Tue, Jan 10, 2012 at 3:16 AM, Simon Riggs wrote: > So benchmarking write-heavy workloads and separately benchmarking > read-only workloads is more representative. Absolutely. High write activity applications are much more difficult to optimize with simple tricks like client side caching. Als

Re: [HACKERS] pgsphere

2012-01-10 Thread Dave Cramer
Hi Oleg, On Sun, Jan 8, 2012 at 1:19 PM, Oleg Bartunov wrote: > Dave, > > The situation with pgshpere is so, that I think we need new developer, since > Janko keep silence :)  I wrote him several time, since I wanted > pgsphere now could benefit very much from our KNNGiST feature. This is > numb

Re: [HACKERS] Sending notifications from the master to the standby

2012-01-10 Thread Joachim Wieland
On Tue, Jan 10, 2012 at 12:00 AM, Tom Lane wrote: > So this design is non-optimal both for existing uses and for the > proposed new uses, which means nobody will like it.  You could > ameliorate #1 by adding a GUC that determines whether NOTIFY actually > writes WAL, but that's pretty ugly.  In an

[HACKERS] [v9.2] Add GUC sepgsql.client_label

2012-01-10 Thread Kohei KaiGai
This patch adds a new GUC "sepgsql.client_label" that allows client process to switch its privileges into another one, as long as the system security policy admits this transition. Because of this feature, I ported two permissions from "process" class of SELinux; "setcurrent" and "dyntransition". T

Re: [HACKERS] [PATCH] Allow breaking out of hung connection attempts

2012-01-10 Thread Heikki Linnakangas
On 09.01.2012 15:49, Ryan Kelly wrote: On Mon, Jan 09, 2012 at 10:35:50AM +0200, Heikki Linnakangas wrote: That assumes that it's safe to longjmp out of PQconnectdbParams at any instant. It's not. I'm guessing because it could result in a resource leak? Yes, and other unfinished business, too

Re: [HACKERS] Page Checksums

2012-01-10 Thread Benedikt Grundmann
On 10/01/12 09:07, Simon Riggs wrote: > > You can repeat that argument ad infinitum. Even if the CRC covers all the > > pages in the OS buffer cache, it still doesn't cover the pages in the > > shared_buffers, CPU caches, in-transit from one memory bank to another etc. > > You have to draw the line

Re: [HACKERS] LWLOCK_STATS

2012-01-10 Thread Simon Riggs
On Tue, Jan 10, 2012 at 12:24 AM, Jim Nasby wrote: > IIRC, pg_bench is *extremely* write-heavy. There's probably not that many > systems that operate that way. I suspect that most OLTP systems read more > than they write, and some probably have as much as a 10-1 ratio. IMHO the main PostgreSQL

Re: [HACKERS] Generate call graphs in run-time

2012-01-10 Thread Joel Jacobson
Is this only to print out the stack upon errors? Looks like the stack is in the variable error_context_stack. Is it always available containing all the parent functions, even when there is no error? Can I reach it from within pgstat.c? 2012/1/10 Jim Nasby > On Jan 9, 2012, at 2:08 PM, Joel Jacob

Re: [HACKERS] Page Checksums

2012-01-10 Thread Simon Riggs
On Tue, Jan 10, 2012 at 8:04 AM, Heikki Linnakangas wrote: > On 10.01.2012 02:12, Jim Nasby wrote: >> >> Filesystem CRCs very likely will not happen to data that's in the cache. >> For some users, that's a huge amount of data to leave un-protected. > > > You can repeat that argument ad infinitum.

Re: [HACKERS] Sending notifications from the master to the standby

2012-01-10 Thread Simon Riggs
On Tue, Jan 10, 2012 at 5:00 AM, Tom Lane wrote: > Joachim Wieland writes: >> [ send NOTIFYs to slaves by means of: ] Good idea. > I wonder whether it'd be practical to not involve WAL per se in this > at all, but to transmit NOTIFY messages by having walsender processes > follow the notify str

Re: [HACKERS] Page Checksums

2012-01-10 Thread Heikki Linnakangas
On 10.01.2012 02:12, Jim Nasby wrote: Filesystem CRCs very likely will not happen to data that's in the cache. For some users, that's a huge amount of data to leave un-protected. You can repeat that argument ad infinitum. Even if the CRC covers all the pages in the OS buffer cache, it still d