Re: pgbench cpu overhead (was Re: [HACKERS] lazy vxid locks, v1)

2011-07-24 Thread Stefan Kaltenbrunner
On 07/24/2011 03:50 AM, Jeff Janes wrote: > On Mon, Jun 13, 2011 at 7:03 AM, Stefan Kaltenbrunner > wrote: >> On 06/13/2011 01:55 PM, Stefan Kaltenbrunner wrote: >> >> [...] >> >>> all those tests are done with pgbench running on the same box - which >>> has a noticable impact on the results becau

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-07-24 Thread Martin Pihlak
On 07/16/2011 12:46 AM, Tom Lane wrote: > I think the direction to move in ought to be to use the existing buffer > as-is, and have pqCheckOutBufferSpace refuse to enlarge the buffer while > we are in "write frozen" state. It should be OK to append data to the > buffer, though, so long as we remem

Re: pgbench cpu overhead (was Re: [HACKERS] lazy vxid locks, v1)

2011-07-24 Thread Tom Lane
Jeff Janes writes: > How was this profile generated? I get a similar profile using > --enable-profiling and gprof, but I find it not believable. The > complete absence of any calls to libpq is not credible. I don't know > about your profiler, but with gprof they should be listed in the call > g

Re: pgbench cpu overhead (was Re: [HACKERS] lazy vxid locks, v1)

2011-07-24 Thread Tom Lane
Stefan Kaltenbrunner writes: > interesting - iirc we actually had some reports about current libpq > behaviour causing scaling issues on some OSes - see > http://archives.postgresql.org/pgsql-hackers/2009-06/msg00748.php and > some related threads. Iirc the final patch for that was never applied >

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-07-24 Thread Tom Lane
Martin Pihlak writes: > On 07/16/2011 12:46 AM, Tom Lane wrote: >> I think the direction to move in ought to be to use the existing buffer >> as-is, and have pqCheckOutBufferSpace refuse to enlarge the buffer while >> we are in "write frozen" state. It should be OK to append data to the >> buffer

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-07-24 Thread Tom Lane
I wrote: > So this does look like it'd fix the issue for SSL_write, without needing > to introduce a concept of a "write frozen" buffer state. I am wondering > though how far back support for this flag exists in OpenSSL, A bit of archaeology reveals that the flag was introduced in OpenSSL 0.9.4,

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-07-24 Thread Tom Lane
I wrote: > Still wondering about the SSL_read end of it, though. And on that front, some digging around in the OpenSSL source code indicates that they do all their work in internal buffers, and transfer data into SSL_read's result buffer only when ready to return it. So the claim in the documentat

Re: [HACKERS] XPATH vs. server_encoding != UTF-8

2011-07-24 Thread Florian Pflug
On Jul24, 2011, at 01:25 , Florian Pflug wrote: > On Jul23, 2011, at 22:49 , Peter Eisentraut wrote: > >> On lör, 2011-07-23 at 17:49 +0200, Florian Pflug wrote: >>> The current thread about JSON and the ensuing discussion about the >>> XML types' behaviour in non-UTF8 databases made me try out ho

Re: pgbench cpu overhead (was Re: [HACKERS] lazy vxid locks, v1)

2011-07-24 Thread Stefan Kaltenbrunner
On 07/24/2011 05:55 PM, Tom Lane wrote: > Stefan Kaltenbrunner writes: >> interesting - iirc we actually had some reports about current libpq >> behaviour causing scaling issues on some OSes - see >> http://archives.postgresql.org/pgsql-hackers/2009-06/msg00748.php and >> some related threads. Iir

Re: Initial Review: JSON contrib modul was: Re: [HACKERS] Another swing at JSON

2011-07-24 Thread Florian Pflug
On Jul24, 2011, at 05:14 , Robert Haas wrote: > On Fri, Jul 22, 2011 at 10:36 PM, Joey Adams > wrote: >> Interesting. This leads to a couple more questions: >> >> * Should the JSON data type (eventually) have an equality operator? > > +1. +1. >> * Should the JSON input function alphabetize

Re: [HACKERS] Problem with pg_upgrade's directory write check on Windows

2011-07-24 Thread Alvaro Herrera
Excerpts from Bruce Momjian's message of dom jul 24 01:46:08 -0400 2011: > Robert Haas wrote: > > > Should I fix this in pg_upgrade 9.1 for Windows or just in 9.2? ?The > > > check works fine on non-Windows. > > > > Seems worth back-patching to me. > > Attached patch applied and backpatched to 9

[HACKERS] vacuumlo patch

2011-07-24 Thread Tim
Please consider adding this minor patch, or offering improvements. *Problem*: vacuumlo required PostgreSQL to use more locks per transaction than possible resulting in an error and a log file full of "ignored until end of transaction" warnings. (max_locks_per_transaction is limited by shmmax which

[HACKERS] SSL-mode error reporting in libpq

2011-07-24 Thread Tom Lane
In testing the fix for the SSL problem that Martin Pihlak reported, I realized that libpq doesn't really cope very well with errors reported by OpenSSL. In the case at hand, SSL_write returns an SSL_ERROR_SSL code, which pqsecure_write quite reasonably handles by putting "SSL error: bad write retr

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-07-24 Thread Tom Lane
Martin Pihlak writes: > If possible I would like the fix to be backported as well. This is > quite a nasty bug and difficult to track down. Especially as the > actual SSL error messages are masked by the "server closed the > connection unexpectedly" message. I've applied the simplified fix (just

Re: [HACKERS] Problem with pg_upgrade's directory write check on Windows

2011-07-24 Thread Bruce Momjian
Alvaro Herrera wrote: > Excerpts from Bruce Momjian's message of dom jul 24 01:46:08 -0400 2011: > > Robert Haas wrote: > > > > > Should I fix this in pg_upgrade 9.1 for Windows or just in 9.2? ?The > > > > check works fine on non-Windows. > > > > > > Seems worth back-patching to me. > > > > Att

Re: Initial Review: JSON contrib modul was: Re: [HACKERS] Another swing at JSON

2011-07-24 Thread Joey Adams
On Sat, Jul 23, 2011 at 11:14 PM, Robert Haas wrote: > I doubt you're going to want to reinvent TOAST, ... I was thinking about making it efficient to access or update foo.a.b.c.d[1000] in a huge JSON tree. Simply TOASTing the varlena text means we have to unpack the entire datum to access and u

[HACKERS] python cleanup

2011-07-24 Thread Andrew Dunstan
On 04/24/2011 07:31 PM, Peter Eisentraut wrote: On sön, 2011-04-24 at 12:25 -0400, Tom Lane wrote: This file is in fundamental violation of the first commandment of Postgres #includes, which is "thou shalt have no other gods before c.h". We need to put postgres.h *before* the Python.h include.

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-07-24 Thread Peter Geoghegan
On 24 July 2011 21:33, Tom Lane wrote: > I've applied the simplified fix (just set SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER) > as well as a patch to improve the error reporting situation. I'm not exactly sure why, and don't have time to investigate right now, but this commit (fee476da952a1f02f7ccf6e23

Re: Initial Review: JSON contrib modul was: Re: [HACKERS] Another swing at JSON

2011-07-24 Thread Florian Pflug
On Jul25, 2011, at 00:48 , Joey Adams wrote: > On Sun, Jul 24, 2011 at 2:19 PM, Florian Pflug wrote: >> On Jul24, 2011, at 05:14 , Robert Haas wrote: >>> On Fri, Jul 22, 2011 at 10:36 PM, Joey Adams >>> wrote: ... Fortunately, JSON's definition of a "number" is its decimal syntax, so t

Re: [HACKERS] libpq SSL with non-blocking sockets

2011-07-24 Thread Andrew Dunstan
On 07/24/2011 07:49 PM, Peter Geoghegan wrote: On 24 July 2011 21:33, Tom Lane wrote: I've applied the simplified fix (just set SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER) as well as a patch to improve the error reporting situation. I'm not exactly sure why, and don't have time to investigate right

Re: [HACKERS] vacuumlo patch

2011-07-24 Thread Alvaro Herrera
Excerpts from Tim's message of dom jul 24 14:48:08 -0400 2011: > Please consider adding this minor patch, or offering improvements. > > *Problem*: vacuumlo required PostgreSQL to use more locks per transaction > than possible resulting in an error and a log file full of "ignored until > end of tra

Re: [HACKERS] Policy on pulling in code from other projects?

2011-07-24 Thread Alvaro Herrera
Excerpts from Dave Page's message of sáb jul 23 02:25:30 -0400 2011: > Also consider if the library is widely available on common distros or > not. If not, packagers are going to have to start packaging that > first, in order to build the PostgreSQL packages. This is a *huge* > issue for use if we

Re: [HACKERS] python cleanup

2011-07-24 Thread Tom Lane
Andrew Dunstan writes: > On my Linux system the attached compiles without warnings. If this seems > like the way to go I'll investigate more on Windows. Hmm ... > +/* > + * Save settings the Python headers might override > + */ > +#ifdef _POSIX_C_SOURCE > +#define _PGSAVE_POSIX_C_SOURCE _POSIX

Re: [HACKERS] vacuumlo patch

2011-07-24 Thread Tim
Hi Álvaro, thanks for the response. Here is the requested "diff with 3 lines of context": git diff HEAD -- contrib/vacuumlo/vacuumlo.c diff --git a/contrib/vacuumlo/vacuumlo.c b/contrib/vacuumlo/vacuumlo.c index f6e2a28..b7c7d64 100644 --- a/contrib/vacuumlo/vacuumlo.c +++ b/contrib/vacuumlo/vacu

Re: [HACKERS] Tracing in Postgres

2011-07-24 Thread Harshitha S
I want to retain all the error messages, error report that is used by Postgres. I don't intend to log any information extra other than what is provided by Postgres. But I just want to replace the implementation of the logging/tracing in Postgres, so that the existing messages can be redirected to a

Re: Initial Review: JSON contrib modul was: Re: [HACKERS] Another swing at JSON

2011-07-24 Thread Joey Adams
On Sun, Jul 24, 2011 at 2:19 PM, Florian Pflug wrote: > The downside being that we'd then either need to canonicalize in > the equality operator, or live with either no equality operator or > a rather strange one. It just occurred to me that, even if we sort object members, texteq might not be a

Re: Initial Review: JSON contrib modul was: Re: [HACKERS] Another swing at JSON

2011-07-24 Thread Joey Adams
On Mon, Jul 25, 2011 at 1:05 AM, Joey Adams wrote: > Should we mimic IEEE floats and preserve -0 versus +0 while treating > them as equal?  Or should we treat JSON floats like numeric and > convert -0 to 0 on input?  Or should we do something else?  I think > converting -0 to 0 would be a bad idea