Re: COPY FROM WITH HEADER skips a tuple every 4 billion tuples

2018-05-22 Thread Tom Lane
Andres Freund writes: > On 2018-05-23 09:04:35 +1200, David Rowley wrote: >> I thought the output I pasted was clearly showing it not to be the >> same. 42 vs 43. > Well, the row-returned counter is obviously wide enough, otherwise > 42 couldn't be returned. Tom's point, a

Re: COPY FROM WITH HEADER skips a tuple every 4 billion tuples

2018-05-22 Thread David Rowley
On 23 May 2018 at 09:31, Andres Freund wrote: >> On 23 May 2018 at 03:55, Tom Lane wrote: >> > Hm, so why is the correct rowcount returned --- are we running >> > a separate counter for that purpose, and if so why? >> >> I thought the output I pasted was clearly showing it not to be the >> same.

Re: COPY FROM WITH HEADER skips a tuple every 4 billion tuples

2018-05-22 Thread Andres Freund
On 2018-05-23 09:04:35 +1200, David Rowley wrote: > Thanks for pushing. > > On 23 May 2018 at 03:55, Tom Lane wrote: > > Hm, so why is the correct rowcount returned --- are we running > > a separate counter for that purpose, and if so why? > > I thought the output I pasted was clearly showing it

Re: COPY FROM WITH HEADER skips a tuple every 4 billion tuples

2018-05-22 Thread David Rowley
On 23 May 2018 at 09:16, Vik Fearing wrote: > I think Tom was wondering why it isn't showing 5032703. You'll need to explain that one. The number just looks like nonsense to me. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Serv

Re: COPY FROM WITH HEADER skips a tuple every 4 billion tuples

2018-05-22 Thread Vik Fearing
On 22/05/18 23:04, David Rowley wrote: > Thanks for pushing. > > On 23 May 2018 at 03:55, Tom Lane wrote: >> Hm, so why is the correct rowcount returned --- are we running >> a separate counter for that purpose, and if so why? > > I thought the output I pasted was clearly showing it not to be th

Re: COPY FROM WITH HEADER skips a tuple every 4 billion tuples

2018-05-22 Thread David Rowley
Thanks for pushing. On 23 May 2018 at 03:55, Tom Lane wrote: > Hm, so why is the correct rowcount returned --- are we running > a separate counter for that purpose, and if so why? I thought the output I pasted was clearly showing it not to be the same. 42 vs 43. Did I misunderst

Re: COPY FROM WITH HEADER skips a tuple every 4 billion tuples

2018-05-22 Thread Tom Lane
Andres Freund writes: > On 2018-05-22 11:55:26 -0400, Tom Lane wrote: >> Hm, so why is the correct rowcount returned --- are we running >> a separate counter for that purpose, and if so why? > Yes, it's a local counter in CopyFrom/CopyTo. It's probably not > entirely trivial to unify the two. Th

Re: COPY FROM WITH HEADER skips a tuple every 4 billion tuples

2018-05-22 Thread Andres Freund
On 2018-05-22 11:55:26 -0400, Tom Lane wrote: > David Rowley writes: > > while it might not look too scary by itself, it gets a bit more so > > when you learn that the cur_lineno is only 32 bits wide. This will > > result in skipping a tuple every time the 32-bit variable wraps back > > around to

Re: COPY FROM WITH HEADER skips a tuple every 4 billion tuples

2018-05-22 Thread Tom Lane
David Rowley writes: > while it might not look too scary by itself, it gets a bit more so > when you learn that the cur_lineno is only 32 bits wide. This will > result in skipping a tuple every time the 32-bit variable wraps back > around to 0 again. Hm, so why is the correct rowcount returned --