Re: [HACKERS] walprotocol.h vs frontends

2011-08-16 Thread Simon Riggs
On Tue, Aug 16, 2011 at 9:35 AM, Magnus Hagander wrote: >> So right now what we do is allow a single packet to span multiple >> files, but since MAX_SEND_SIZE is 128KB it will always be smaller than >> a single file, so we can only ever span two files at most. > > Unless someone has tweaked their

Re: [HACKERS] walprotocol.h vs frontends

2011-08-16 Thread Peter Eisentraut
On mån, 2011-08-15 at 18:39 +0100, Peter Geoghegan wrote: > > If you want to upgrade a system running 8.3 (that uses float based > timestamps) in using > > pg_upgrade you must compile 9.0 (or 8.4 or 9.1) with > > --disable-integer-datetimes. If at some point in the future you > then want > > to up

Re: [HACKERS] walprotocol.h vs frontends

2011-08-16 Thread Magnus Hagander
On Tue, Aug 16, 2011 at 00:05, Simon Riggs wrote: > On Mon, Aug 15, 2011 at 10:32 PM, Magnus Hagander wrote: > >>> At present the WALSender only sends from one file at a time, so >>> sending a message when we open a new file would be straightforward. >> >> Are you sure? We can receive a single me

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Simon Riggs
On Mon, Aug 15, 2011 at 10:32 PM, Magnus Hagander wrote: >> At present the WALSender only sends from one file at a time, so >> sending a message when we open a new file would be straightforward. > > Are you sure? We can receive a single message spanning multiple files... You're right. That was t

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Dimitri Fontaine
Magnus Hagander writes: >> Why not have a specific protocol message to indicate a change of filename? >> I don't mean a WAL message, I mean a streaming protocol message. > > That we could have, and it would work as long as it's always sent as > the first packet in any stream. > > If we do that, we

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Magnus Hagander
On Mon, Aug 15, 2011 at 21:12, Simon Riggs wrote: > On Mon, Aug 15, 2011 at 5:15 PM, Magnus Hagander wrote: >> On Mon, Aug 15, 2011 at 18:12, Tom Lane wrote: >>> Heikki Linnakangas writes: Perhaps we should change the protocol so that it explicitly says which file the streamed piece o

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Simon Riggs
On Mon, Aug 15, 2011 at 5:15 PM, Magnus Hagander wrote: > On Mon, Aug 15, 2011 at 18:12, Tom Lane wrote: >> Heikki Linnakangas writes: >>> Perhaps we should change the protocol so that it explicitly says which >>> file the streamed piece of WAL belongs to. That way a client could write >>> it to

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Peter Geoghegan
On 15 August 2011 18:09, Steve Singer wrote: >> Really? I find that slightly surprising, considering that a quick look >> at master's timestamp.c suggests that the choice to use the in64 >> representation over the double representation is entirely a case of >> compile time either/or. There is no a

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Tom Lane
Steve Singer writes: > On 11-08-15 12:33 PM, Peter Geoghegan wrote: >> On 15 August 2011 16:56, Steve Singer wrote: >>> This would mean that anyone using the floating point timestamps today won't >>> be able to use pg_upgrade to upgrade to whichever version we remove them >>> from. 8.3 had float

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Steve Singer
On 11-08-15 12:33 PM, Peter Geoghegan wrote: On 15 August 2011 16:56, Steve Singer wrote: This would mean that anyone using the floating point timestamps today won't be able to use pg_upgrade to upgrade to whichever version we remove them from. 8.3 had float based timestamps as the default and

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Magnus Hagander
On Mon, Aug 15, 2011 at 18:40, Tom Lane wrote: > Magnus Hagander writes: >> On Mon, Aug 15, 2011 at 16:53, Tom Lane wrote: >>> Don't think you should expose fsec_t, nor most of those macros.  The >>> foo_per_bar values are just namespace clutter. > >> Hmm, ok. I just went for what seemed like a

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Tom Lane
Magnus Hagander writes: > On Mon, Aug 15, 2011 at 16:53, Tom Lane wrote: >> Don't think you should expose fsec_t, nor most of those macros.  The >> foo_per_bar values are just namespace clutter. > Hmm, ok. I just went for what seemed like a reasonable subset. I do > also need the SECS_PER_DAY an

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Peter Geoghegan
On 15 August 2011 16:56, Steve Singer wrote: > This would mean that anyone using the floating point timestamps today won't > be able to use pg_upgrade to upgrade to whichever version we remove them > from.  8.3 had float based timestamps as the default and I suspect many > installations with the d

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Magnus Hagander
On Mon, Aug 15, 2011 at 18:12, Tom Lane wrote: > Heikki Linnakangas writes: >> Perhaps we should change the protocol so that it explicitly says which >> file the streamed piece of WAL belongs to. That way a client could write >> it to the correct file without knowing about all those macros. > > Y

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Tom Lane
Heikki Linnakangas writes: > Perhaps we should change the protocol so that it explicitly says which > file the streamed piece of WAL belongs to. That way a client could write > it to the correct file without knowing about all those macros. Yeah, maybe. Otherwise, all that logic is not only imp

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Heikki Linnakangas
On 15.08.2011 18:54, Magnus Hagander wrote: On Mon, Aug 15, 2011 at 16:53, Tom Lane wrote: Magnus Hagander writes: I also ran into a similar problem with some WAL macro definitions that are in xlog_internal.h. I've moved them to xlogdefs.h in the attached xlog.diff file. Does that seem ok as

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Steve Singer
On 11-08-15 10:00 AM, Peter Geoghegan wrote: Without commenting on what should be done in your specific case, I wonder whether it's time to fully retire the deprecated double representation of timestamps. Is anyone actually expected to rely on their availability when 9.2 is released? This also c

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Magnus Hagander
On Mon, Aug 15, 2011 at 16:53, Tom Lane wrote: > Magnus Hagander writes: >> On Mon, Aug 15, 2011 at 16:20, Tom Lane wrote: >>> However, for a narrow fix, I could see moving the data type definition >>> to someplace with fewer dependencies.  Perhaps split it into a separate >>> file timestamp_typ

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Tom Lane
Magnus Hagander writes: > On Mon, Aug 15, 2011 at 16:20, Tom Lane wrote: >> However, for a narrow fix, I could see moving the data type definition >> to someplace with fewer dependencies.  Perhaps split it into a separate >> file timestamp_type.h, or something like that. > Yes, that seems to fix

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Magnus Hagander
On Mon, Aug 15, 2011 at 16:20, Tom Lane wrote: > Magnus Hagander writes: >> I'm trying to make my streaming log receiver work properly with 9.1, >> and have come across a couple of things. The first one that's causing >> trouble is that the definition of the protocol is currently in >> walprotoco

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Tom Lane
Magnus Hagander writes: > I'm trying to make my streaming log receiver work properly with 9.1, > and have come across a couple of things. The first one that's causing > trouble is that the definition of the protocol is currently in > walprotocol.h, which is not include:able in a frontend applicati

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Peter Geoghegan
On 15 August 2011 12:22, Magnus Hagander wrote: > The basic reason for this is that we're putting TimestampTz fields in > the protocol. This also means that the protocol actually changes > definition depending on if the server is compiled with integer or > float timestamps. Without commenting on

[HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Magnus Hagander
I'm trying to make my streaming log receiver work properly with 9.1, and have come across a couple of things. The first one that's causing trouble is that the definition of the protocol is currently in walprotocol.h, which is not include:able in a frontend application. AFAICT, this is because it in