Re: [HACKERS] Re: COPY BINARY file format proposal

2000-12-13 Thread Peter Eisentraut
Tom Lane writes: > I take it from the smiley that you're not serious, but actually it seems > like it might not be a bad idea. I could see appending a CRC to each > tuple record. Comments anyone? I think I missed the point here. With CRC you typically want to detect data corruption. Where's

Re: [HACKERS] Re: COPY BINARY file format proposal

2000-12-12 Thread Tom Lane
[EMAIL PROTECTED] (Nathan Myers) writes: > I don't know if you get the point of the fixed-size comment field. > The idea was that a comment could be poked into an existing COPY > image, after it was written. Yes, I did get the point ... > A variable-size comment field in an > already-written

Re: [HACKERS] Re: COPY BINARY file format proposal

2000-12-12 Thread Nathan Myers
On Sun, Dec 10, 2000 at 08:51:52PM -0500, Tom Lane wrote: > [EMAIL PROTECTED] (Nathan Myers) writes: > > I'd like to see a timestamp for when the image was created, and a > > 128-byte comment field to allow annotations, even after the fact. > > Both seem like reasonable options. If you don't mi

Re: [HACKERS] Re: COPY BINARY file format proposal

2000-12-12 Thread Tom Lane
Philip Warner <[EMAIL PROTECTED]> writes: > How about a CRC? ;-P I take it from the smiley that you're not serious, but actually it seems like it might not be a bad idea. I could see appending a CRC to each tuple record. Comments anyone? You seemed to like the PNG philosophy of using feature f

Re: [HACKERS] Re: COPY BINARY file format proposal

2000-12-12 Thread Tom Lane
Philip Warner <[EMAIL PROTECTED]> writes: > More a matter of not thinking it was important enough to worry about, and > not really wanting to drag the MD5/MD4/CRC64/etc debate into this one. I'd just as soon not drag that debate in here either ;-) ... but once we settle on an appropriate CRC meth

Re: [HACKERS] Re: COPY BINARY file format proposal

2000-12-12 Thread Philip Warner
At 19:55 8/12/00 -0500, Tom Lane wrote: >Philip Warner <[EMAIL PROTECTED]> writes: >> How about a CRC? ;-P > >I take it from the smiley that you're not serious, but actually it seems >like it might not be a bad idea. I could see appending a CRC to each >tuple record. Comments anyone? More a mat

Re: [HACKERS] Re: COPY BINARY file format proposal

2000-12-12 Thread Philip Warner
At 01:27 8/12/00 -0500, Tom Lane wrote: >Recovering the data on a machine >of different endianness is a project for future data archeologists. It's frightening to think that in 1000 years time people will be deducing things about our society from the way we stored data. > >Tell you the truth, I

Re: [HACKERS] Re: COPY BINARY file format proposal

2000-12-10 Thread Bruce Momjian
> [EMAIL PROTECTED] (Nathan Myers) writes: > > I'd like to see a timestamp for when the image was created, and a > > 128-byte comment field to allow annotations, even after the fact. > > Both seem like reasonable options. If you don't mind, however, > I'd suggest that they be left for inclusion

Re: [HACKERS] Re: COPY BINARY file format proposal

2000-12-10 Thread Tom Lane
[EMAIL PROTECTED] (Nathan Myers) writes: > I'd like to see a timestamp for when the image was created, and a > 128-byte comment field to allow annotations, even after the fact. Both seem like reasonable options. If you don't mind, however, I'd suggest that they be left for inclusion as chunks i

Re: [HACKERS] Re: COPY BINARY file format proposal

2000-12-10 Thread Ross J. Reedstrom
On Sun, Dec 10, 2000 at 04:08:58PM -0800, Nathan Myers wrote: > On Thu, Dec 07, 2000 at 02:28:28PM -0500, Tom Lane wrote: > > Given all that, here is a proposed spec for the header: > > ... > > Comments? > > (I have also suggested, in private mail, that the "header length" > field should be the

Re: [HACKERS] Re: COPY BINARY file format proposal

2000-12-10 Thread Bruce Momjian
> Also, IIRC part of the problem with text-based COPY is that we can't > specify field order (I think this affectes dumping the regression DB). > Would it be possible to add the ability to (a) specify field order, and (b) > dump a subset of fields? Informix does this nicely: UNLOAD TO "f

Re: [HACKERS] Re: COPY BINARY file format proposal

2000-12-10 Thread Nathan Myers
On Thu, Dec 07, 2000 at 02:28:28PM -0500, Tom Lane wrote: > Given all that, here is a proposed spec for the header: > ... > Comments? I've been thinking about this. I'd like to see a timestamp for when the image was created, and a 128-byte comment field to allow annotations, even after the f

Re: [HACKERS] Re: COPY BINARY file format proposal

2000-12-10 Thread Bruce Momjian
> I will not complain about sticking a "version 1.0" field into a format > when there is no real intention of changing it in the future ... but > assigning deep significance to major/minor numbers, or something like I assume the version would be the COPY format version, not the PostgreSQL version

Re: [HACKERS] Re: COPY BINARY file format proposal

2000-12-08 Thread Philip Warner
At 02:31 8/12/00 -0500, Tom Lane wrote: > >How about: > >Signature: 12-byte sequence "PGBCOPY\n\377\r\n\0" (detects newline >replacements, dropped nulls, dropped high bits, parity changes); > >Integer layout field: int32 constant 0x01020304 in source's byte order. > How about a CRC? ;-P ---

Re: [HACKERS] Re: COPY BINARY file format proposal

2000-12-07 Thread Tom Lane
I wrote: > Next 4 bytes: integer layout field. This consists of the int32 constant > 0x0A820D0A expressed in the source machine's endianness. (Again, value > chosen with malice aforethought, to catch files munged by things like > DOS/Unix newline conversion or high-bit-stripping.) Actually, tha

Re: [HACKERS] Re: COPY BINARY file format proposal

2000-12-07 Thread Tom Lane
Philip Warner <[EMAIL PROTECTED]> writes: > I don't want to continue being picky, but you could just use 4 bytes for a > maj-min-rev-patch version number (in that order), and avoid the endian > issues by reading and writing each byte. No big deal, though. Well, the thing is that we need to protec

Re: [HACKERS] Re: COPY BINARY file format proposal

2000-12-07 Thread Philip Warner
At 14:28 7/12/00 -0500, Tom Lane wrote: > >Next 4 bytes: version number, currently 1 (expressed in source machine's >endianness I don't want to continue being picky, but you could just use 4 bytes for a maj-min-rev-patch version number (in that order), and avoid the endian issues by reading and w

Re: [HACKERS] Re: COPY BINARY file format proposal

2000-12-07 Thread Tom Lane
Philip Warner <[EMAIL PROTECTED]> writes: >> Just thinking that the only way an endianness flag inside the header >> would be useful is if we pick a magic number that's a bytewise >> palindrome. > You could just read the 1st, 2nd, 3rd, etc bytes and require that they be > 'P', 'G', 'C', 'P', 'Y'

Re: [HACKERS] Re: COPY BINARY file format proposal

2000-12-06 Thread Philip Warner
At 15:36 6/12/00 -0500, Tom Lane wrote: >Grumble, I forgot about COPY WITH OIDS. Amend that proposal as follows: > >... We should use two different >magic numbers depending on whether OIDs are included in the dump or not. I'd prefer to see a single magic number for all binary COPY output, then a

Re: [HACKERS] Re: COPY BINARY file format proposal

2000-12-06 Thread Philip Warner
At 21:12 6/12/00 -0500, Tom Lane wrote: >Philip Warner <[EMAIL PROTECTED]> writes: >>> OK, we can do it that way. I'm still going to pick a magic number that >>> looks different depending on endianness, however ;-). > >> What does the smiley mean in this context? > >Just thinking that the only wa

Re: [HACKERS] Re: COPY BINARY file format proposal

2000-12-06 Thread Tom Lane
Philip Warner <[EMAIL PROTECTED]> writes: >> OK, we can do it that way. I'm still going to pick a magic number that >> looks different depending on endianness, however ;-). > What does the smiley mean in this context? Just thinking that the only way an endianness flag inside the header would be

Re: [HACKERS] Re: COPY BINARY file format proposal

2000-12-06 Thread Philip Warner
At 20:40 6/12/00 -0500, Tom Lane wrote: >Philip Warner <[EMAIL PROTECTED]> writes: >> I'd prefer to see a single magic number for all binary COPY output, then a >> few bytes of header including a version number, and flags to indicate >> endianness, OIDs etc. It seems a lot cleaner than overloading

Re: [HACKERS] Re: COPY BINARY file format proposal

2000-12-06 Thread Tom Lane
Philip Warner <[EMAIL PROTECTED]> writes: > I'd prefer to see a single magic number for all binary COPY output, then a > few bytes of header including a version number, and flags to indicate > endianness, OIDs etc. It seems a lot cleaner than overloading the magic > number. OK, we can do it that