Re: Race condition in recovery?

2021-05-17 Thread Kyotaro Horiguchi
At Tue, 18 May 2021 15:52:07 +0900 (JST), Kyotaro Horiguchi wrote in > FWIW, you could be get a problematic base backup by the following steps. > > 0. (make sure /tmp/hoge is removed) > 1. apply the attached patch > 2. create a primary then start > 3. create a standby then start > 4. place stan

Re: Race condition in recovery?

2021-05-17 Thread Kyotaro Horiguchi
At Mon, 17 May 2021 10:46:24 +0530, Dilip Kumar wrote in > On Mon, May 17, 2021 at 10:09 AM Dilip Kumar wrote: > > > > On Mon, May 17, 2021 at 8:50 AM Kyotaro Horiguchi > > wrote: > > > > > > Before the commit expectedTLEs is always initialized with just one > > > entry for the TLI of the last

Re: pg_get_wal_replay_pause_state() should not return 'paused' while a promotion is ongoing.

2021-05-17 Thread Michael Paquier
On Tue, May 18, 2021 at 12:48:38PM +0900, Fujii Masao wrote: > Currently a promotion causes all available WAL to be replayed before > a standby becomes a primary whether it was in paused state or not. > OTOH, something like immediate promotion (i.e., standby becomes > a primary without replaying ou

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-17 Thread Amit Langote
On Mon, May 17, 2021 at 9:45 PM osumi.takami...@fujitsu.com wrote: > On Monday, May 17, 2021 6:52 PM Amit Langote wrote: > > Both patches are attached. > The patch for PG13 can be applied to it cleanly and the RT succeeded. > > I have few really minor comments on your comments in the patch. > > (

Re: subscriptioncheck failure

2021-05-17 Thread vignesh C
On Tue, May 18, 2021 at 9:09 AM Amit Kapila wrote: > > On Mon, May 17, 2021 at 5:48 PM vignesh C wrote: > > > > On Mon, May 17, 2021 at 10:40 AM Amit Kapila > > wrote: > > > > > > One more point: > > > + $node_publisher->wait_for_catchup('tap_sub'); > > > + > > > + # Ensure a transactional logi

Re: pg_get_wal_replay_pause_state() should not return 'paused' while a promotion is ongoing.

2021-05-17 Thread Dilip Kumar
On Mon, May 17, 2021 at 7:59 PM Fujii Masao wrote: > > If a promotion is triggered while recovery is paused, the paused state ends > and promotion continues. But currently pg_get_wal_replay_pause_state() > returns 'paused' in that case. Isn't this a bug? > > Attached patch fixes this issue by rese

Re: Testing autovacuum wraparound (including failsafe)

2021-05-17 Thread Masahiko Sawada
On Tue, May 18, 2021 at 2:42 PM Peter Geoghegan wrote: > > On Mon, May 17, 2021 at 10:29 PM Masahiko Sawada > wrote: > > +1 to fix this. Are you already working on fixing this? If not, I'll > > post a patch. > > I posted a patch recently (last Thursday my time). Perhaps you can review it? Oh, I

Re: Testing autovacuum wraparound (including failsafe)

2021-05-17 Thread Peter Geoghegan
On Mon, May 17, 2021 at 10:29 PM Masahiko Sawada wrote: > +1 to fix this. Are you already working on fixing this? If not, I'll > post a patch. I posted a patch recently (last Thursday my time). Perhaps you can review it? -- Peter Geoghegan

Re: Race condition in recovery?

2021-05-17 Thread Dilip Kumar
On Tue, May 18, 2021 at 1:28 AM Robert Haas wrote: > > Sorry, you're right. It couldn't be uninitialized, but it could be a > fake 1-element list saying there are no ancestors rather than the real > value. So I think the point was to avoid that. Yeah, it will be a fake 1-element list. But just t

Re: Testing autovacuum wraparound (including failsafe)

2021-05-17 Thread Masahiko Sawada
On Sat, Apr 24, 2021 at 11:16 AM Peter Geoghegan wrote: > > On Fri, Apr 23, 2021 at 5:29 PM Andres Freund wrote: > > On 2021-04-23 16:12:33 -0700, Peter Geoghegan wrote: > > > The only reason that I chose 4GB for FAILSAFE_MIN_PAGES is because the > > > related VACUUM_FSM_EVERY_PAGES constant was

Re: "ERROR: deadlock detected" when replicating TRUNCATE

2021-05-17 Thread Amit Kapila
On Tue, May 18, 2021 at 6:19 AM Peter Smith wrote: > > On Mon, May 17, 2021 at 8:13 PM Amit Kapila wrote: > > > > On Mon, May 17, 2021 at 3:05 PM Dilip Kumar wrote: > > > > > > On Mon, May 17, 2021 at 12:30 PM Peter Smith > > > wrote: > > > > > > > PSA a patch adding a test for this scenario.

RE: Bug in query rewriter - hasModifyingCTE not getting set

2021-05-17 Thread tsunakawa.ta...@fujitsu.com
From: Tom Lane > In view of this, maybe the right thing is to disallow modifying CTEs > in rule actions in the first place. I see we already do that for > views (i.e. ON SELECT rules), but they're not really any safer in > other types of rules. You meant by views something like the following, di

Re: pg_get_wal_replay_pause_state() should not return 'paused' while a promotion is ongoing.

2021-05-17 Thread Fujii Masao
On 2021/05/18 9:58, Kyotaro Horiguchi wrote: At Mon, 17 May 2021 23:29:18 +0900, Fujii Masao wrote in If a promotion is triggered while recovery is paused, the paused state ends and promotion continues. But currently pg_get_wal_replay_pause_state() returns 'paused' in that case. Isn't this

Re: subscriptioncheck failure

2021-05-17 Thread Amit Kapila
On Mon, May 17, 2021 at 5:48 PM vignesh C wrote: > > On Mon, May 17, 2021 at 10:40 AM Amit Kapila wrote: > > > > One more point: > > + $node_publisher->wait_for_catchup('tap_sub'); > > + > > + # Ensure a transactional logical decoding message shows up on the slot > > + $node_subscriber->safe_psql

Re: Always bump PG_CONTROL_VERSION?

2021-05-17 Thread Michael Paquier
On Mon, May 17, 2021 at 03:47:01PM -0700, Andres Freund wrote: > There's several types of tools (particularly around backup) that need to > parse control files. Unnecessarily increasing the numbers of versions > that need to be dealt with makes that a bit harder. I am digressing here, sorry for th

Re: Skip partition tuple routing with constant partition key

2021-05-17 Thread Michael Paquier
On Tue, May 18, 2021 at 01:27:48PM +1200, David Rowley wrote: > A slightly different optimization that I have considered and even > written patches before was to have ExecFindPartition() cache the last > routed to partition and have it check if the new row can go into that > one on the next call.

Re: What is lurking in the shadows?

2021-05-17 Thread David Rowley
On Tue, 18 May 2021 at 14:09, Peter Smith wrote: > Yeah, I would like to work my way through all of these warnings in my > spare time and report back to this thread (after 1-2 months?) with a > detailed analysis. I'd recommend for any patches that they come in bite-sized chunks. A committer is go

Re: What is lurking in the shadows?

2021-05-17 Thread Michael Paquier
On Tue, May 18, 2021 at 12:08:57PM +1000, Peter Smith wrote: > Yeah, I would like to work my way through all of these warnings in my > spare time and report back to this thread (after 1-2 months?) with a > detailed analysis. The next commit fest is at the beginning of July, so there are a couple o

Re: Addition of authenticated ID to pg_stat_activity

2021-05-17 Thread Michael Paquier
On Mon, May 17, 2021 at 10:28:49AM +0200, Magnus Hagander wrote: > On Mon, May 17, 2021 at 6:35 AM Michael Paquier wrote: >> Not sure if we would be able to agree on something here, but the >> barrier to what a session and a connection hold is thin when it comes >> to roles and application_name.

Re: Performance degradation of REFRESH MATERIALIZED VIEW

2021-05-17 Thread Masahiko Sawada
On Wed, May 12, 2021 at 2:32 AM Tomas Vondra wrote: > > > > On 5/11/21 5:56 PM, Masahiko Sawada wrote: > > On Tue, May 11, 2021 at 11:07 PM Tomas Vondra > > wrote: > >> > >> On 5/11/21 11:04 AM, Masahiko Sawada wrote: > >>> On Tue, May 11, 2021 at 4:37 PM Michael Paquier > >>> wrote: > > >

RE: Skip partition tuple routing with constant partition key

2021-05-17 Thread houzj.f...@fujitsu.com
> > Hmm, does this seem common enough for the added complexity to be > worthwhile? > > I'd also like to know if there's some genuine use case for this. For testing > purposes does not seem to be quite a good enough reason. Thanks for the response. For some big data scenario, we sometimes transfe

Re: What is lurking in the shadows?

2021-05-17 Thread Peter Smith
On Fri, May 14, 2021 at 11:16 AM David Rowley wrote: > > On Fri, 14 May 2021 at 12:00, Peter Smith wrote: > > That bug led me to wonder if similar problems might be going > > undetected elsewhere in the code. There is a gcc compiler option [3] > > -Wshadow which informs about the similar scenario

Re: Rewriting the test of pg_upgrade as a TAP test - take three - remastered set

2021-05-17 Thread Michael Paquier
On Mon, May 17, 2021 at 12:32:13PM -0400, Andrew Dunstan wrote: > On 5/16/21 9:55 PM, Michael Paquier wrote: > Yes, I'm going to be proposing a series of smallish patches including > these when the tree is branched (which I hope will be in a few weeks). Thanks! That clearly needs to happen first.

Re: allow specifying direct role membership in pg_hba.conf

2021-05-17 Thread Chapman Flack
On 05/17/21 21:19, Chapman Flack wrote: > This makes twice in a row that I've failed to see how. > > If you go through the entries, in order, and simply prune from the list > the ones you can already prove would never apply to this connection, how > does that break the ordering principle? Ok, I

Re: Skip partition tuple routing with constant partition key

2021-05-17 Thread David Rowley
On Tue, 18 May 2021 at 01:31, Amit Langote wrote: > Hmm, does this seem common enough for the added complexity to be worthwhile? I'd also like to know if there's some genuine use case for this. For testing purposes does not seem to be quite a good enough reason. A slightly different optimization

RE: Re: Parallel scan with SubTransGetTopmostTransaction assert coredump

2021-05-17 Thread Pengchengliu
Hi Greg, > I actually think that the Assert in SubTransGetTopmostTransaction() is correct, but in the parallel-worker case, the snapshots are not being setup correctly. I agree with you that Assert in SubTransGetTopmostTransaction() is correct. The root cause is Transaction Xmin are not

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-17 Thread Michael Paquier
On Mon, May 17, 2021 at 02:28:57PM -0700, Andres Freund wrote: > On 2021-05-17 17:06:32 -0400, Tom Lane wrote: >> Putting it just after attalign seems like a reasonably sane choice >> from the standpoint of grouping things affecting physical storage; >> and as you say, that wins from the standpoint

Re: "ERROR: deadlock detected" when replicating TRUNCATE

2021-05-17 Thread Peter Smith
On Mon, May 17, 2021 at 6:47 PM Amit Kapila wrote: > > On Mon, May 17, 2021 at 12:30 PM Peter Smith wrote: > > [...] > > The essence of the trouble seems to be that the apply_handle_truncate > > function never anticipated it may end up truncating the same table > > from 2 separate workers (subscr

Re: allow specifying direct role membership in pg_hba.conf

2021-05-17 Thread Chapman Flack
On 05/17/21 17:55, Tom Lane wrote: > This seems pretty horrid to me, not only from a complexity standpoint, > but because it would break the principle that pg_hba.conf entries are > applied in order. This makes twice in a row that I've failed to see how. If you go through the entries, in order, a

Re: pg_get_wal_replay_pause_state() should not return 'paused' while a promotion is ongoing.

2021-05-17 Thread Kyotaro Horiguchi
At Mon, 17 May 2021 23:29:18 +0900, Fujii Masao wrote in > If a promotion is triggered while recovery is paused, the paused state > ends > and promotion continues. But currently pg_get_wal_replay_pause_state() > returns 'paused' in that case. Isn't this a bug? > > Attached patch fixes this issu

Re: wal stats questions

2021-05-17 Thread Masahiro Ikeda
On 2021/05/17 22:34, Fujii Masao wrote: > > > On 2021/05/17 16:39, Masahiro Ikeda wrote: >> >> Thanks for your comments! >> + * is executed, wal records aren't nomally generated (although HOT makes >>> >>> nomally -> normally? >> >> Yes, fixed. >> + * It's not enough to

Re: "ERROR: deadlock detected" when replicating TRUNCATE

2021-05-17 Thread Peter Smith
On Mon, May 17, 2021 at 8:13 PM Amit Kapila wrote: > > On Mon, May 17, 2021 at 3:05 PM Dilip Kumar wrote: > > > > On Mon, May 17, 2021 at 12:30 PM Peter Smith wrote: > > > > > PSA a patch adding a test for this scenario. > > > > I am not sure this test case is exactly targeting the problematic >

Re: pg_dumpall misses --no-toast-compression

2021-05-17 Thread Michael Paquier
On Mon, May 17, 2021 at 10:38:15AM -0400, Tom Lane wrote: > Daniel Gustafsson writes: >> I think there is a reasonable case to be made for this fixing an oversight in >> bbe0a81db69bd10bd166907c3701492a29aca294 as opposed to adding a brand new >> feature. Save for --no-synchronized-snapshots all

Re: Typo in README.barrier

2021-05-17 Thread Michael Paquier
On Mon, May 17, 2021 at 06:37:56PM -0400, Tom Lane wrote: > Generally it's considered poor form to push any inessential patches > during a release window (which I'd define roughly as 48 hours before > the wrap till after the tag is applied). It complicates the picture > for the final round of buil

Re: Always bump PG_CONTROL_VERSION?

2021-05-17 Thread Andres Freund
Hi, On 2021-05-13 17:42:52 -0400, Jan Wieck wrote: > Also, can someone give me a good reason NOT to bump the version? There's several types of tools (particularly around backup) that need to parse control files. Unnecessarily increasing the numbers of versions that need to be dealt with makes tha

Re: Typo in README.barrier

2021-05-17 Thread Tom Lane
David Rowley writes: >>> You may want to hold on until 14beta1 is tagged, though. >> Of course we can wait till that day but I wonder why. > I imagined that would be a good idea for more risky patches so we > don't break something before a good round of buildfarm testing. > However, since this i

Re: Typo in README.barrier

2021-05-17 Thread David Rowley
On Mon, 17 May 2021 at 17:18, Tatsuo Ishii wrote: > > > On Mon, May 17, 2021 at 09:33:27AM +0900, Tatsuo Ishii wrote: > >> Me too. Let's backpatch. > > > > A README is not directly user-facing, it is here for developers, so I > > would not really bother with a backpatch. Now it is not a big deal

Re: allow specifying direct role membership in pg_hba.conf

2021-05-17 Thread Tom Lane
Chapman Flack writes: > On 05/17/21 16:35, Magnus Hagander wrote: >> So you're saying that some entries int he parameter section would >> depend on the db/user/ip combo and some would depend just on the ip? > I don't *think* that's what I was saying. What I was thinking was this: > ... This seem

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-17 Thread Andres Freund
Hi, On 2021-05-17 17:06:32 -0400, Tom Lane wrote: > Putting it just after attalign seems like a reasonably sane choice > from the standpoint of grouping things affecting physical storage; > and as you say, that wins from the standpoint of using up alignment > padding rather than adding more. Make

Re: allow specifying direct role membership in pg_hba.conf

2021-05-17 Thread Chapman Flack
On 05/17/21 16:35, Magnus Hagander wrote: > So you're saying that some entries int he parameter section would > depend on the db/user/ip combo and some would depend just on the ip? I don't *think* that's what I was saying. What I was thinking was this: The pg_hba.conf file is an ordered list of e

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-17 Thread Tom Lane
Andres Freund writes: > If we moved attcompression to all the other bool/char fields, we'd avoid > that size increase, as there's an existing 2 byte hole. +1. Looks to me like its existing placement was according to the good old "add new things at the end" anti-pattern. It certainly isn't relat

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-17 Thread Justin Pryzby
On Mon, May 17, 2021 at 01:48:03PM -0700, Andres Freund wrote: > pg_attribute is one of the biggest table in a new cluster, and often the > biggest table in production clusters. Its size is also quite relevant in > memory, due to all the TupleDescs we allocate. > > I just noticed that the new attc

Re: ICU bool problem

2021-05-17 Thread Bruce Momjian
On Mon, May 17, 2021 at 10:56:54PM +0200, Peter Eisentraut wrote: > The fix is like what we used to use for plperl back then: > > diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h > index f3e04d4d8c..499ada2b69 100644 > --- a/src/include/utils/pg_locale.h > +++ b/src/inclu

ICU bool problem

2021-05-17 Thread Peter Eisentraut
You might find that ICU 69 (pretty new, see http://site.icu-project.org/download/69) will cause compile failures with PG 10 (pretty old). ICU 69 has switched to using stdbool.h, which conflicts with the home-made definitions that we used until PG10. Compile errors look like this: pg_collatio

Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-17 Thread Andres Freund
Hi, pg_attribute is one of the biggest table in a new cluster, and often the biggest table in production clusters. Its size is also quite relevant in memory, due to all the TupleDescs we allocate. I just noticed that the new attcompression increased the size not just by 1 byte, but by 4, due to p

Re: allow specifying direct role membership in pg_hba.conf

2021-05-17 Thread Magnus Hagander
On Mon, May 17, 2021 at 10:31 PM Chapman Flack wrote: > > On 05/17/21 16:15, Magnus Hagander wrote: > > The row is selected by the combination of username/database/ipaddress. > > But you have to pick the minimum TLS version before the client has > > sent that... Basically we have to make the choic

Re: PG 14 release notes, first draft

2021-05-17 Thread Bruce Momjian
On Mon, May 17, 2021 at 12:45:44PM -0700, Peter Geoghegan wrote: > I propose that the B-Tree index item (bottom-up index deletion, "Allow > btree index additions to remove expired index entries to prevent page > splits") link back to the documentation for those that want to drill > down: > > https

Re: allow specifying direct role membership in pg_hba.conf

2021-05-17 Thread Chapman Flack
On 05/17/21 16:15, Magnus Hagander wrote: > The row is selected by the combination of username/database/ipaddress. > But you have to pick the minimum TLS version before the client has > sent that... Basically we have to make the choice long before we've > even started looking at pg_hba. Use the pe

Re: allow specifying direct role membership in pg_hba.conf

2021-05-17 Thread Magnus Hagander
On Fri, May 14, 2021 at 8:58 PM Stephen Frost wrote: > > Greetings, > > * Chapman Flack (c...@anastigmatix.net) wrote: > > If pg_hba syntax changes are being entertained, I would love to be able > > to set ssl_min_protocol_version locally in a hostssl rule. > > > > Some clients at $work are stuck

Re: pg_hba.conf.sample wording improvement

2021-05-17 Thread Magnus Hagander
On Thu, Apr 29, 2021 at 5:06 PM Stephen Frost wrote: > > Greetings, > > * Magnus Hagander (mag...@hagander.net) wrote: > > On Thu, Apr 29, 2021 at 7:08 AM Peter Eisentraut > > wrote: > > > On 28.04.21 16:09, Alvaro Herrera wrote: > > > > Looking at it now, I wonder how well do the "hostno" option

Re: Race condition in recovery?

2021-05-17 Thread Robert Haas
On Sat, May 15, 2021 at 1:25 AM Dilip Kumar wrote: > > As I understand it, the general issue here was that if > > XLogFileReadAnyTLI() was called before expectedTLEs got set, then > > prior to this commit it would have to fail, because the foreach() loop > > in that function would be iterating ov

Re: Winflex docs and distro

2021-05-17 Thread Magnus Hagander
On Mon, May 17, 2021 at 2:55 PM Andrew Dunstan wrote: > > > On 5/17/21 5:51 AM, Magnus Hagander wrote: > > On Mon, May 17, 2021 at 11:11 AM Daniel Gustafsson wrote: > >>> On 17 May 2021, at 10:17, Magnus Hagander wrote: > >>> Since PostgreSQL 9.3, in commit a266f7dd93b, we've added the text: > >

Re: PG 14 release notes, first draft

2021-05-17 Thread Peter Geoghegan
On Mon, May 17, 2021 at 12:45 PM Peter Geoghegan wrote: > https://www.postgresql.org/docs/14/btree-implementation.html#BTREE-DELETION This linke currently 404s -- https://www.postgresql.org/docs/devel/btree-implementation.html#BTREE-DELETION works, though. -- Peter Geoghegan

Re: PG 14 release notes, first draft

2021-05-17 Thread Peter Geoghegan
I propose that the B-Tree index item (bottom-up index deletion, "Allow btree index additions to remove expired index entries to prevent page splits") link back to the documentation for those that want to drill down: https://www.postgresql.org/docs/14/btree-implementation.html#BTREE-DELETION The s

Re: PG 14 release notes, first draft

2021-05-17 Thread Alvaro Herrera
On 2021-May-16, Bruce Momjian wrote: > OK, what I was trying to say was that if you dump/restore, and the old > password was md5, the newly-restored password will be md5, but it was > very unclear. I changed it to this: > > > > > > Change the default of the pass

Re: prion failed with ERROR: missing chunk number 0 for toast value 14334 in pg_toast_2619

2021-05-17 Thread Andres Freund
Hi, On 2021-05-17 20:14:40 +0200, Drouvot, Bertrand wrote: > FWIW a patch proposal to copy the oldest unfrozen XID during pg_upgrade (it > adds a new (- u) parameter to pg_resetwal) has been submitted a couple of > weeks ago, see: https://commitfest.postgresql.org/33/3105/ I'll try to look at it

Re: Rewriting the test of pg_upgrade as a TAP test - take three - remastered set

2021-05-17 Thread Andrew Dunstan
On 5/16/21 9:55 PM, Michael Paquier wrote: > On Sat, May 15, 2021 at 02:22:24PM -0400, Andrew Dunstan wrote: >> PostgresNode is currently able to create nodes suitable for upgrade down >> to release 10. If we add '-w' to the 'pg_ctl start' flags that can >> extend down to release 9.5. (Just teste

Re: [HACKERS] logical decoding of two-phase transactions

2021-05-17 Thread vignesh C
On Mon, May 17, 2021 at 6:10 PM Ajin Cherian wrote: > > The above patch had some changes missing which resulted in some tap > tests failing. Sending an updated patchset. Keeping the patchset > version the same. Thanks for the updated patch, the updated patch fixes the tap test failures. Regards,

Re: Removed extra memory allocations from create_list_bounds

2021-05-17 Thread Justin Pryzby
On Mon, May 17, 2021 at 08:22:25PM +0530, Nitin Jadhav wrote: > I agree and thanks for creating those patches. I am not able to apply > the patch on the latest HEAD. Kindly check and upload the modified > patches. The CFBOT had no issues with the patches, so I suspect an issue on your side. http:/

Re: Removed extra memory allocations from create_list_bounds

2021-05-17 Thread Zhihong Yu
On Mon, May 17, 2021 at 7:52 AM Nitin Jadhav wrote: > > > While working on [1], I observed that extra memory is allocated in > > > [1] > https://mail.google.com/mail/u/2/#search/multi+column+list/KtbxLxgZZTjRxNrBWvmHzDTHXCHLssSprg?compose=CllgCHrjDqKgWCBNMmLqhzKhmrvHhSRlRVZxPCVcLkLmFQwrccpTpqLNgb

Re: Corrected documentation of data type for the logical replication message formats.

2021-05-17 Thread vignesh C
On Thu, May 13, 2021 at 5:57 AM Peter Smith wrote: > > On Wed, May 12, 2021 at 11:09 PM vignesh C wrote: > ... > > > > Thanks for the comments. Attached v4 patch has the fix for the same. > > > > I have not tried this patch so I cannot confirm whether it applies or > renders OK, but just going by

Re: Removed extra memory allocations from create_list_bounds

2021-05-17 Thread Nitin Jadhav
> > While working on [1], I observed that extra memory is allocated in > > [1] > > https://mail.google.com/mail/u/2/#search/multi+column+list/KtbxLxgZZTjRxNrBWvmHzDTHXCHLssSprg?compose=CllgCHrjDqKgWCBNMmLqhzKhmrvHhSRlRVZxPCVcLkLmFQwrccpTpqLNgbWqKkTkTFCHMtZjWnV I am really sorry for this. I wanted

Re: pg_dumpall misses --no-toast-compression

2021-05-17 Thread Tom Lane
Daniel Gustafsson writes: >> On 17 May 2021, at 03:12, Michael Paquier wrote: >> I have bumped into $subject while playing with this feature, and this >> can be really useful to be able to reset the compression method for >> all the tables at restore. The patch is simple but that's perhaps too >

pg_get_wal_replay_pause_state() should not return 'paused' while a promotion is ongoing.

2021-05-17 Thread Fujii Masao
If a promotion is triggered while recovery is paused, the paused state ends and promotion continues. But currently pg_get_wal_replay_pause_state() returns 'paused' in that case. Isn't this a bug? Attached patch fixes this issue by resetting the recovery pause state to 'not paused' when standby pr

Re: postgres_fdw - should we tighten up batch_size, fetch_size options against non-numeric values?

2021-05-17 Thread Bharath Rupireddy
On Mon, May 17, 2021 at 6:17 PM Ashutosh Bapat wrote: > > On Mon, May 17, 2021 at 3:29 PM Bharath Rupireddy > wrote: > > > > Hi, > > > > It looks like the values such as '123.456', '789.123' '100$%$#$#', > > '9,223,372,' are accepted and treated as valid integers for > > postgres_fdw options batc

Re: PG 14 release notes, first draft

2021-05-17 Thread Jonathan S. Katz
Hi, On 5/10/21 2:03 AM, Bruce Momjian wrote: > I have committed the first draft of the PG 14 release notes. You can > see the most current build of them here: > > https://momjian.us/pgsql_docs/release-14.html > > I need clarification on many items, and the document still needs its > item

Re: pg_dumpall misses --no-toast-compression

2021-05-17 Thread Daniel Gustafsson
> On 17 May 2021, at 03:12, Michael Paquier wrote: > I have bumped into $subject while playing with this feature, and this > can be really useful to be able to reset the compression method for > all the tables at restore. The patch is simple but that's perhaps too > late for 14, so I am adding i

Re: PG 14 release notes, first draft

2021-05-17 Thread Bruce Momjian
On Mon, May 17, 2021 at 04:18:27PM +0900, Fujii Masao wrote: > > > On 2021/05/10 15:03, Bruce Momjian wrote: > > I have committed the first draft of the PG 14 release notes. You can > > see the most current build of them here: > > > > https://momjian.us/pgsql_docs/release-14.html > > Than

Re: wal stats questions

2021-05-17 Thread Fujii Masao
On 2021/05/17 16:39, Masahiro Ikeda wrote: Thanks for your comments! + * is executed, wal records aren't nomally generated (although HOT makes nomally -> normally? Yes, fixed. + * It's not enough to check the number of generated wal records, for + * example the walwriter m

Re: Skip partition tuple routing with constant partition key

2021-05-17 Thread Amit Langote
On Mon, May 17, 2021 at 8:37 PM houzj.f...@fujitsu.com wrote: > When loading some data into a partitioned table for testing purpose, > > I found even if I specified constant value for the partition key[1], it still > do > > the tuple routing for each row. > > > [1]- > > UPDATE

Re: Re: Parallel scan with SubTransGetTopmostTransaction assert coredump

2021-05-17 Thread Greg Nancarrow
On Sat, May 15, 2021 at 12:37 PM 刘鹏程 wrote: > > > BTW, I test it in a high performance server. It is verly easily be > reproduced. My colleague and me use different environment both can reproduce > it. > Hi Pengcheng, Although the issue won't reproduce easily in my system, I can certainly se

Re: Winflex docs and distro

2021-05-17 Thread Andrew Dunstan
On 5/17/21 5:51 AM, Magnus Hagander wrote: > On Mon, May 17, 2021 at 11:11 AM Daniel Gustafsson wrote: >>> On 17 May 2021, at 10:17, Magnus Hagander wrote: >>> Since PostgreSQL 9.3, in commit a266f7dd93b, we've added the text: >>> >>> + The obsolete "winflex" binaries distributed on the P

Re: postgres_fdw - should we tighten up batch_size, fetch_size options against non-numeric values?

2021-05-17 Thread Ashutosh Bapat
On Mon, May 17, 2021 at 3:29 PM Bharath Rupireddy wrote: > > Hi, > > It looks like the values such as '123.456', '789.123' '100$%$#$#', > '9,223,372,' are accepted and treated as valid integers for > postgres_fdw options batch_size and fetch_size. Whereas this is not > the case with fdw_startup_co

RE: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-17 Thread osumi.takami...@fujitsu.com
On Monday, May 17, 2021 6:52 PM Amit Langote wrote: > On Mon, May 17, 2021 at 6:13 PM Amit Kapila > wrote: > > On Fri, May 14, 2021 at 12:44 PM Amit Langote > wrote: > > > On Thu, May 13, 2021 at 7:43 PM Amit Kapila > wrote: > > > > > > > Also, don't we need to free the > > > > entire map as su

Re: subscriptioncheck failure

2021-05-17 Thread vignesh C
On Mon, May 17, 2021 at 10:40 AM Amit Kapila wrote: > > On Mon, May 17, 2021 at 9:06 AM Amit Kapila wrote: > > > > On Thu, May 13, 2021 at 7:06 PM vignesh C wrote: > > > > > > On Thu, May 13, 2021 at 4:41 PM Michael Paquier > > > wrote: > > > > Few comments: > > 1. > > + # Ensure a transaction

Skip partition tuple routing with constant partition key

2021-05-17 Thread houzj.f...@fujitsu.com
Hi, When loading some data into a partitioned table for testing purpose, I found even if I specified constant value for the partition key[1], it still do the tuple routing for each row. [1]- UPDATE partitioned set part_key = 2 , … INSERT into partitioned(part_key, ...)

Re: Possible memory corruption (src/timezone/zic.c b/src/timezone/zic.c)

2021-05-17 Thread Ranier Vilela
Em dom., 16 de mai. de 2021 às 22:37, Kyotaro Horiguchi < horikyota@gmail.com> escreveu: > At Sat, 15 May 2021 11:35:13 -0300, Ranier Vilela > wrote in > > Em sex., 14 de mai. de 2021 às 19:52, Tom Lane > escreveu: > > > > > I wrote: > > > > So the question for us is whether it's worth tryin

RE: Parallel scan with SubTransGetTopmostTransaction assert coredump

2021-05-17 Thread Pengchengliu
Hi Tom & Robert, Could you review this Assert(TransactionIdFollowsOrEquals(xid, TransactionXmin)) in SubTransGetTopmostTransaction. I think this assert is unsuitable for parallel work process. Before we discuss it in https://www.postgresql-archive.org/Parallel-scan-with-SubTransGetTopmo

Re: Multiple hosts in connection string failed to failover in non-hot standby mode

2021-05-17 Thread Daniel Gustafsson
> On 11 May 2021, at 09:29, Michael Paquier wrote: > FWIW, I think that the case of getting some information about any > failed connections while a connection has been successfully made > within the scope of the connection string parameters provided by the > user is rather thin, and I really feel

Re: Inaccurate error message when set fdw batch_size to 0

2021-05-17 Thread Amit Kapila
On Tue, May 11, 2021 at 11:28 AM Bharath Rupireddy wrote: > > On Mon, May 10, 2021 at 7:39 PM Tom Lane wrote: > > > > Bharath Rupireddy writes: > > > On Mon, May 10, 2021 at 12:00 PM Tom Lane wrote: > > >> Yeah, this error message seems outright buggy. However, it's a minor > > >> matter. Als

Re: "ERROR: deadlock detected" when replicating TRUNCATE

2021-05-17 Thread Dilip Kumar
On Mon, May 17, 2021 at 3:43 PM Amit Kapila wrote: > > On Mon, May 17, 2021 at 3:05 PM Dilip Kumar wrote: > > > > On Mon, May 17, 2021 at 12:30 PM Peter Smith wrote: > > > > > PSA a patch adding a test for this scenario. > > > > I am not sure this test case is exactly targeting the problematic >

Re: "ERROR: deadlock detected" when replicating TRUNCATE

2021-05-17 Thread Amit Kapila
On Mon, May 17, 2021 at 3:05 PM Dilip Kumar wrote: > > On Mon, May 17, 2021 at 12:30 PM Peter Smith wrote: > > > PSA a patch adding a test for this scenario. > > I am not sure this test case is exactly targeting the problematic > behavior because that will depends upon the order of execution of t

postgres_fdw - should we tighten up batch_size, fetch_size options against non-numeric values?

2021-05-17 Thread Bharath Rupireddy
Hi, It looks like the values such as '123.456', '789.123' '100$%$#$#', '9,223,372,' are accepted and treated as valid integers for postgres_fdw options batch_size and fetch_size. Whereas this is not the case with fdw_startup_cost and fdw_tuple_cost options for which an error is thrown. Attaching a

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-17 Thread Amit Langote
On Mon, May 17, 2021 at 6:13 PM Amit Kapila wrote: > On Fri, May 14, 2021 at 12:44 PM Amit Langote wrote: > > On Thu, May 13, 2021 at 7:43 PM Amit Kapila wrote: > > > > > Also, don't we need to free the > > > entire map as suggested by me? > > > > Yes, I had missed that too. Addressed in the up

Re: Winflex docs and distro

2021-05-17 Thread Magnus Hagander
On Mon, May 17, 2021 at 11:11 AM Daniel Gustafsson wrote: > > > On 17 May 2021, at 10:17, Magnus Hagander wrote: > > > Since PostgreSQL 9.3, in commit a266f7dd93b, we've added the text: > > > > + The obsolete "winflex" binaries distributed on the PostgreSQL FTP > > site > > Which was sligh

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-17 Thread Amit Kapila
On Fri, May 14, 2021 at 2:20 PM osumi.takami...@fujitsu.com wrote: > > On Thursday, May 13, 2021 7:21 PM Amit Kapila wrote: > > I don't think we can reproduce it with core plugins as they don't lock user > > catalog tables. > OK. My current understanding about how the deadlock happens is below. >

RE: "ERROR: deadlock detected" when replicating TRUNCATE

2021-05-17 Thread tanghy.f...@fujitsu.com
On Monday, May 17, 2021 5:47 PM, Amit Kapila wrote > +$node_publisher->safe_psql('postgres', > + "ALTER SYSTEM SET synchronous_standby_names TO 'any 2(sub5_1, > sub5_2)'"); > +$node_publisher->safe_psql('postgres', "SELECT pg_reload_conf()"); > > Do you really need these steps to reproduce the pr

Re: "ERROR: deadlock detected" when replicating TRUNCATE

2021-05-17 Thread Dilip Kumar
On Mon, May 17, 2021 at 12:30 PM Peter Smith wrote: > The essence of the trouble seems to be that the apply_handle_truncate > function never anticipated it may end up truncating the same table > from 2 separate workers (subscriptions) like this test case is doing. > Probably this is quite an old

RE: making update/delete of inheritance trees scale better

2021-05-17 Thread houzj.f...@fujitsu.com
> On Mon, May 17, 2021 at 3:07 PM houzj.f...@fujitsu.com > wrote: > > > > Hi > > > > After 86dc900, In " src/include/nodes/pathnodes.h ", I noticed that it > > uses the word " partitioned UPDATE " in the comment above struct > RowIdentityVarInfo. > > > > But, it seems " inherited UPDATE " is used

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-17 Thread Amit Kapila
On Fri, May 14, 2021 at 12:44 PM Amit Langote wrote: > > On Thu, May 13, 2021 at 7:43 PM Amit Kapila wrote: > > > Also, don't we need to free the > > entire map as suggested by me? > > Yes, I had missed that too. Addressed in the updated patch. > + relentry->map = convert_tuples_by_name(indesc,

Re: Winflex docs and distro

2021-05-17 Thread Daniel Gustafsson
> On 17 May 2021, at 10:17, Magnus Hagander wrote: > Since PostgreSQL 9.3, in commit a266f7dd93b, we've added the text: > > + The obsolete "winflex" binaries distributed on the PostgreSQL FTP site Which was slightly updated in 0a9ae44288d. > At this point. I suggest we simply stop distri

Re: logical replication worker accesses catalogs in error context callback

2021-05-17 Thread Bharath Rupireddy
On Fri, Apr 16, 2021 at 3:23 PM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > > On Tue, Mar 16, 2021 at 2:21 AM Tom Lane wrote: > > > > Bharath Rupireddy writes: > > > Thanks for pointing to the changes in the commit message. I corrected > > > them. Attaching v4 patch set,

Re: Schema variables - new implementation for Postgres 15

2021-05-17 Thread Pavel Stehule
> > Pavel > > >> Regards >> >> Pavel >> > schema-variables-20210517.patch.gz Description: application/gzip

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-05-17 Thread Magnus Hagander
On Fri, Apr 23, 2021 at 12:40 PM Fujii Masao wrote: > > > > On 2021/04/23 19:11, Magnus Hagander wrote: > > On Fri, Apr 23, 2021 at 12:04 PM Fujii Masao > > wrote: > >> > >> > >> > >> On 2021/04/23 18:46, Magnus Hagander wrote: > >>> On Fri, Apr 23, 2021 at 9:10 AM Fujii Masao > >>> wrote: > >>

Re: "ERROR: deadlock detected" when replicating TRUNCATE

2021-05-17 Thread Amit Kapila
On Mon, May 17, 2021 at 12:30 PM Peter Smith wrote: > > While doing logical replication testing we encountered a problem which > causes a deadlock error to be logged when replicating a TRUNCATE > simultaneously to 2 Subscriptions. > e.g. > -- > 2021-05-12 11:30:19.457 AEST [11393] ERROR:

Re: Asynchronous Append on postgres_fdw nodes.

2021-05-17 Thread Etsuro Fujita
On Sun, May 16, 2021 at 11:39 PM Etsuro Fujita wrote: > Attached is a new version. I have committed the patch. Best regards, Etsuro Fujita

Re: FDW and connections

2021-05-17 Thread Phil Godfrin
Apologies, in my mind this was an internals to the postgres_fdw code, which is why I cam here. I checked that part of the docs and nowhere does it say anything about defaulting to 5432. However in the referred section,  34.1.2, there it says that libpq defaults to the "port number established w

Re: Addition of authenticated ID to pg_stat_activity

2021-05-17 Thread Magnus Hagander
On Mon, May 17, 2021 at 6:35 AM Michael Paquier wrote: > > On Thu, Apr 29, 2021 at 04:56:42PM +0200, Magnus Hagander wrote: > > I definitely use it all the time to monitor autovacuum all the time. > > The others as well regularly, but autovacuum continuously. I also see > > a lot of people doing t

Winflex docs and distro

2021-05-17 Thread Magnus Hagander
Since PostgreSQL 9.3, in commit a266f7dd93b, we've added the text: + The obsolete "winflex" binaries distributed on the PostgreSQL FTP site + and referenced in older documentation will fail with "flex: fatal + internal error, exec failed" on 64-bit Windows hosts. Use flex from +

Re: Different compression methods for FPI

2021-05-17 Thread Michael Paquier
On Sun, Mar 21, 2021 at 02:30:04PM -0500, Justin Pryzby wrote: > rebased to keep cfbot happy. This will run with default=zlib. I have been looking at bit at this patch set, and I think that we should do something here for 15~. First, I think that we should make more tests and pick up one compres

  1   2   >