Re: [SPAM] Re: [GENERAL] Invalid field size

2017-07-05 Thread Moreno Andreo
Il 05/07/2017 16:33, Adrian Klaver ha scritto: On 07/05/2017 01:05 AM, Moreno Andreo wrote: Il 04/07/2017 20:51, Daniel Verite ha scritto: Tom Lane wrote: Moreno Andreo writes: So the hint is to abandon manual COPY and let pg_dump do the hard work? If it is a newline-conversion problem,

Re: [GENERAL] Invalid field size

2017-07-05 Thread Adrian Klaver
On 07/05/2017 01:05 AM, Moreno Andreo wrote: Il 04/07/2017 20:51, Daniel Verite ha scritto: Tom Lane wrote: Moreno Andreo writes: So the hint is to abandon manual COPY and let pg_dump do the hard work? If it is a newline-conversion problem, compressed pg_dump archives would be just as s

Re: [GENERAL] Invalid field size

2017-07-05 Thread Moreno Andreo
Il 04/07/2017 20:51, Daniel Verite ha scritto: Tom Lane wrote: Moreno Andreo writes: So the hint is to abandon manual COPY and let pg_dump do the hard work? If it is a newline-conversion problem, compressed pg_dump archives would be just as subject to corruption as your binary COPY f

Re: [SPAM] Re: [SPAM] Re: [GENERAL] Invalid field size

2017-07-04 Thread Adrian Klaver
On 07/04/2017 10:57 AM, Moreno Andreo wrote: Il 04/07/2017 19:28, Tom Lane ha scritto: Moreno Andreo writes: So the hint is to abandon manual COPY and let pg_dump do the hard work? If it is a newline-conversion problem, compressed pg_dump archives would be just as subject to corruption as you

Re: [GENERAL] Invalid field size

2017-07-04 Thread Tom Lane
"Daniel Verite" writes: > Tom Lane wrote: >> If it is a newline-conversion problem, compressed pg_dump archives would >> be just as subject to corruption as your binary COPY file is. > It's mentioned in [1] that the signature at the beginning of these files > embed a CRLF to detect this ne

Re: [GENERAL] Invalid field size

2017-07-04 Thread Daniel Verite
Tom Lane wrote: > Moreno Andreo writes: > > So the hint is to abandon manual COPY and let pg_dump do the hard work? > > If it is a newline-conversion problem, compressed pg_dump archives would > be just as subject to corruption as your binary COPY file is. It's mentioned in [1] that th

Re: [SPAM] Re: [SPAM] Re: [GENERAL] Invalid field size

2017-07-04 Thread Moreno Andreo
Il 04/07/2017 19:28, Tom Lane ha scritto: Moreno Andreo writes: So the hint is to abandon manual COPY and let pg_dump do the hard work? If it is a newline-conversion problem, compressed pg_dump archives would be just as subject to corruption as your binary COPY file is. I'd say the hint is to

Re: [SPAM] Re: [SPAM] Re: [GENERAL] Invalid field size

2017-07-04 Thread Tom Lane
Moreno Andreo writes: > So the hint is to abandon manual COPY and let pg_dump do the hard work? If it is a newline-conversion problem, compressed pg_dump archives would be just as subject to corruption as your binary COPY file is. I'd say the hint is to be more careful about how you do the cross

Re: [SPAM] Re: [SPAM] Re: [GENERAL] Invalid field size

2017-07-04 Thread Adrian Klaver
On 07/04/2017 10:13 AM, Moreno Andreo wrote: Il 04/07/2017 18:25, Adrian Klaver ha scritto: On 07/04/2017 09:02 AM, Moreno Andreo wrote: Il 04/07/2017 17:39, Adrian Klaver ha scritto: So what you are saying is "in the last 5 years you've been extremely lucky?" :-) Your original post went b

Re: [SPAM] Re: [SPAM] Re: [GENERAL] Invalid field size

2017-07-04 Thread Moreno Andreo
Il 04/07/2017 18:25, Adrian Klaver ha scritto: On 07/04/2017 09:02 AM, Moreno Andreo wrote: Il 04/07/2017 17:39, Adrian Klaver ha scritto: So what you are saying is "in the last 5 years you've been extremely lucky?" :-) Your original post went back and forth on whether you where lucky in t

Re: [GENERAL] Invalid field size

2017-07-04 Thread Moreno Andreo
Il 04/07/2017 18:55, Daniel Verite ha scritto: I don't quite see from your posts whether that particular file to import was tried and failed only once or retried and failed again. Only once, and until the user will not return from holidays I'll not be able to reproduce it. Cheers Moreno -

Re: [GENERAL] Invalid field size

2017-07-04 Thread Daniel Verite
Moreno Andreo wrote: > So if it's the case (hardware error), recalling a new backup should > reproduce the error, right? If the error happened when writing the file, I wouldn't expect any other backup having the same error (assuming an error in the bit-flip category). And if it was a tr

Re: [SPAM] Re: [GENERAL] Invalid field size

2017-07-04 Thread Adrian Klaver
On 07/04/2017 09:02 AM, Moreno Andreo wrote: Il 04/07/2017 17:39, Adrian Klaver ha scritto: So what you are saying is "in the last 5 years you've been extremely lucky?" :-) Your original post went back and forth on whether you where lucky in the past: "... that's been working well in the

Re: [GENERAL] Invalid field size

2017-07-04 Thread Adrian Klaver
On 07/04/2017 08:37 AM, Moreno Andreo wrote: Il 04/07/2017 17:25, Tom Lane ha scritto: Moreno Andreo writes: Il 04/07/2017 16:51, Tom Lane ha scritto: Pushing binary data around on Windows is always a hazardous proposition. So what you are saying is "in the last 5 years you've been extremely

Re: [GENERAL] Invalid field size

2017-07-04 Thread Moreno Andreo
Il 04/07/2017 17:42, Daniel Verite ha scritto: Moreno Andreo wrote: As you can see I have 2 bytea fields, blob and thumbnail (the one it seems it's giving the error), but AFAIK the former is never used, so it should be always null. Googling around did not help. Despite the auto-correc

Re: [SPAM] Re: [GENERAL] Invalid field size

2017-07-04 Thread Moreno Andreo
Il 04/07/2017 17:42, Adrian Klaver ha scritto: On 07/04/2017 08:37 AM, Moreno Andreo wrote: Il 04/07/2017 17:25, Tom Lane ha scritto: Moreno Andreo writes: Il 04/07/2017 16:51, Tom Lane ha scritto: Pushing binary data around on Windows is always a hazardous proposition. So what you are sayi

Re: [SPAM] Re: [GENERAL] Invalid field size

2017-07-04 Thread Moreno Andreo
Il 04/07/2017 17:39, Adrian Klaver ha scritto: On 07/04/2017 08:19 AM, Moreno Andreo wrote: Il 04/07/2017 16:51, Tom Lane ha scritto: Moreno Andreo writes: I've implemented a backup procedure in C# with Npgsql (using COPY TO I dump all tables in a compressed file) that's been working well in

Re: [GENERAL] Invalid field size

2017-07-04 Thread Daniel Verite
Moreno Andreo wrote: > As you can see I have 2 bytea fields, blob and thumbnail (the one it > seems it's giving the error), but AFAIK the former is never used, so it > should be always null. > Googling around did not help. In COPY BINARY, NULL is represented as -1 (all bits set) in the

Re: [GENERAL] Invalid field size

2017-07-04 Thread Adrian Klaver
On 07/04/2017 08:37 AM, Moreno Andreo wrote: Il 04/07/2017 17:25, Tom Lane ha scritto: Moreno Andreo writes: Il 04/07/2017 16:51, Tom Lane ha scritto: Pushing binary data around on Windows is always a hazardous proposition. So what you are saying is "in the last 5 years you've been extremely

Re: [GENERAL] Invalid field size

2017-07-04 Thread Adrian Klaver
On 07/04/2017 08:19 AM, Moreno Andreo wrote: Il 04/07/2017 16:51, Tom Lane ha scritto: Moreno Andreo writes: I've implemented a backup procedure in C# with Npgsql (using COPY TO I dump all tables in a compressed file) that's been working well in the last 5 years (and it's still working, since

Re: [GENERAL] Invalid field size

2017-07-04 Thread Moreno Andreo
Il 04/07/2017 17:25, Tom Lane ha scritto: Moreno Andreo writes: Il 04/07/2017 16:51, Tom Lane ha scritto: Pushing binary data around on Windows is always a hazardous proposition. So what you are saying is "in the last 5 years you've been extremely lucky?" :-) Yup, particularly now that you m

Re: [GENERAL] Invalid field size

2017-07-04 Thread Tom Lane
Moreno Andreo writes: > Il 04/07/2017 16:51, Tom Lane ha scritto: >> Pushing binary data around on Windows is always a hazardous proposition. > So what you are saying is "in the last 5 years you've been extremely > lucky?" :-) Yup, particularly now that you mention moving the files between mach

Re: [GENERAL] Invalid field size

2017-07-04 Thread Moreno Andreo
Il 04/07/2017 16:51, Tom Lane ha scritto: Moreno Andreo writes: I've implemented a backup procedure in C# with Npgsql (using COPY TO I dump all tables in a compressed file) that's been working well in the last 5 years (and it's still working, since this is a single, isolated case). OS: Windows

Re: [SPAM] Re: [GENERAL] Invalid field size

2017-07-04 Thread Moreno Andreo
Il 04/07/2017 16:36, Adrian Klaver ha scritto: On 07/04/2017 04:16 AM, Moreno Andreo wrote: I've implemented a backup procedure in C# with Npgsql (using COPY TO I dump all tables in a compressed file) that's been working well in the last 5 years (and it's still working, since this is a single,

Re: [GENERAL] Invalid field size

2017-07-04 Thread Tom Lane
Moreno Andreo writes: > I've implemented a backup procedure in C# with Npgsql (using COPY TO I > dump all tables in a compressed file) that's been working well in the > last 5 years (and it's still working, since this is a single, isolated > case). > OS: Windows 7 > PG: 9.1.6 (I know, it's EOL,

Re: [GENERAL] Invalid field size

2017-07-04 Thread Moreno Andreo
Il 04/07/2017 16:30, Glyn Astill ha scritto: >On Tuesday, 4 July 2017, 12:16:57 GMT+1, Moreno Andreo wrote: > > > Any ideas? As for many error I got in the past I assume we are trying to > COPY FROM corrupte

Re: [GENERAL] Invalid field size

2017-07-04 Thread Adrian Klaver
On 07/04/2017 04:16 AM, Moreno Andreo wrote: I've implemented a backup procedure in C# with Npgsql (using COPY TO I dump all tables in a compressed file) that's been working well in the last 5 years (and it's still working, since this is a single, isolated case). OS: Windows 7 PG: 9.1.6 (I kn

Re: [GENERAL] Invalid field size

2017-07-04 Thread Glyn Astill
>On Tuesday, 4 July 2017, 12:16:57 GMT+1, Moreno Andreo > wrote: > > > Any ideas? As for many error I got in the past I assume we are trying to > COPY FROM corrupted data (when using cheap pendrives we get often this > error). Should it be reasonable or I have to search elsewhere? I'd start by