On Wed, Jan 31, 2018 at 12:32 PM, Stefan Blanke < stefan.bla...@framestore.com> wrote:
> > > > I'll bet you it's not that. It's quite unlikely that would fail with > > exactly 1GB request size. It seems much more like a buffer that we keep > > to be power of 2. The question is which one. > > I had dismissed corruption before writing in. It's exactly 1GB every time > this has happened - and we can dump the full dataset periodically without > issue. > > >> I have my money on a corrupted TOAST entry. Is this happening on > >> trustworthy hardware or beige box with no ECC or RAID? > > It's good quality commercial hardware in our colo - no exactly sure what. > If it is a sporadic issue and you can dump the full dataset, then I just lost my money (Tomas, you coming to PGConf in Jersey City?). But then, if this is a plain COPY to stdout ... I am wondering what could possibly be in that path that wants to allocate 1GB. Or is this not so plain but rather a COPY ... SELECT ...? Regards, Jan > > > Thanks for taking the time to look at this! > Stefan > > > > > > On 01/30/18 22:00, Tomas Vondra wrote: > >> >> >> On 01/30/2018 10:43 PM, Jan Wieck wrote: >> >>> >>> >>> On Tue, Jan 30, 2018 at 12:35 PM, Stefan Blanke >>> <stefan.bla...@framestore.com <mailto:stefan.bla...@framestore.com>> >>> wrote: >>> >>> Hello, >>> >>> We've tripped over an error when doing a "COPY.. TO STDOUT WITH >>> BINARY" query. >>> >>> "ERROR: invalid memory alloc request size 1073741824" >>> (exactly 1GB) >>> >>> >>> I have my money on a corrupted TOAST entry. Is this happening on >>> trustworthy hardware or beige box with no ECC or RAID? >>> >>> >> I'll bet you it's not that. It's quite unlikely that would fail with >> exactly 1GB request size. It seems much more like a buffer that we keep >> to be power of 2. The question is which one. >> >> >> regards >> >> -- Jan Wieck Senior Postgres Architect http://pgblog.wi3ck.info
