Re: [HACKERS] logical decoding of two-phase transactions

2021-07-08 Thread Ajin Cherian
On Fri, Jul 9, 2021 at 9:13 AM Peter Smith wrote: > I tried the v95-0001 patch. > > - The patch applied cleanly and all build / testing was OK. > - The documentation also builds OK. > - I checked all v95-0001 / v93-0001 differences and found no problems. > - Furthermore, I noted that v95-0001 pat

Re: Added schema level support for publication.

2021-07-08 Thread Greg Nancarrow
On Fri, Jul 9, 2021 at 1:28 PM houzj.f...@fujitsu.com wrote: > > Currently, postgres caches publication actions info in the > RelationData::rd_pubactions, but after applying the patch, it seems > rd_pubactions is not initialized when using schema level publication. > > It cound result in some unex

Re: [PoC] Improve dead tuple storage for lazy vacuum

2021-07-08 Thread Masahiko Sawada
On Fri, Jul 9, 2021 at 12:53 PM Andres Freund wrote: > > Hi, > > > On 2021-07-07 20:46:38 +0900, Masahiko Sawada wrote: > > 1. Don't allocate more than 1GB. There was a discussion to eliminate > > this limitation by using MemoryContextAllocHuge() but there were > > concerns about point 2[1]. > > >

Re: Transactions involving multiple postgres foreign servers, take 2

2021-07-08 Thread Fujii Masao
On 2021/06/30 10:05, Masahiko Sawada wrote: I've attached the new version patch that incorporates the comments from Fujii-san and Ikeda-san I got so far. Thanks for updating the patches! I'm now reading 0001 and 0002 patches and wondering if we can commit them at first because they just pro

Re: Outdated comments about proc->sem in lwlock.c

2021-07-08 Thread Thomas Munro
On Thu, Jul 8, 2021 at 8:48 AM Daniel Gustafsson wrote: > > On 3 Jun 2021, at 04:07, Thomas Munro wrote: > > Here's a patch to remove the misleading comments. > > While not an expert in the area; reading the referenced commit and the code > with the now removed comments, I think this is correct.

Re: when the startup process doesn't (logging startup delays)

2021-07-08 Thread Amul Sul
Few comments for v4 patch: @@ -7351,6 +7363,8 @@ StartupXLOG(void) (errmsg("redo starts at %X/%X", LSN_FORMAT_ARGS(ReadRecPtr; + InitStartupProgress(); + /* * main redo apply loop */ @@ -7358,6 +73

Re: More time spending with "delete pending"

2021-07-08 Thread Michael Paquier
On Thu, Jul 08, 2021 at 11:00:00PM +0300, Alexander Lakhin wrote: > Beside the aforementioned test I can only propose the extended patch, > that incorporates the undo of the changes brought by bed90759f. > With this patch that test is passed. Checked and confirmed. It is a nice test with IPC::Run

Re: [PoC] Improve dead tuple storage for lazy vacuum

2021-07-08 Thread Andres Freund
Hi, On 2021-07-08 20:53:32 -0700, Andres Freund wrote: > On 2021-07-07 20:46:38 +0900, Masahiko Sawada wrote: > > 1. Don't allocate more than 1GB. There was a discussion to eliminate > > this limitation by using MemoryContextAllocHuge() but there were > > concerns about point 2[1]. > > > > 2. Allo

Re: bugfix: when the blocksize is 32k, the function page_header of pageinspect returns negative numbers.

2021-07-08 Thread Michael Paquier
On Fri, Jul 09, 2021 at 11:11:46AM +0800, Quan Zongliang wrote: > Thanks for the comments. Thanks. Having four switches is a bit repetitive so I would just use one of these, and perhaps complete with some assertions to make sure that atttypid matches to what is expected. That's a minor comment t

Re: pgbench logging broken by time logic changes

2021-07-08 Thread Fabien COELHO
Hello Thomas, Isn't it better if we only have to throw away the first one?). This should be the user decision to drop it or not, not the tool producing it, IMO. Let me try this complaint again. [...] I understand your point. For me removing silently the last bucket is not right because

Re: RFC: Logging plan of the running query

2021-07-08 Thread torikoshia
On 2021-07-02 23:21, Bharath Rupireddy wrote: On Tue, Jun 22, 2021 at 8:00 AM torikoshia wrote: Updated the patch. Thanks for the patch. Here are some comments on the v4 patch: Thanks for your comments and suggestions! I agree with you and updated the patch. On Thu, Jul 1, 2021 at 3:34 PM

Re: enable_resultcache confusion

2021-07-08 Thread Thomas Munro
On Fri, Jul 9, 2021 at 6:03 AM David G. Johnston wrote: > On Thu, Jul 8, 2021 at 11:00 AM Tom Lane wrote: >> Maybe name the plan node type Memoize, and the GUC "enable_memoize"? > > +1 +1

Re: pgbench logging broken by time logic changes

2021-07-08 Thread Thomas Munro
On Fri, Jul 9, 2021 at 5:15 AM Fabien COELHO wrote: > > Thanks! This doesn't seem to address the complaint, though. Don't > > you need to do something like this? (See also attached.) > > > > +initStats(&aggs, start - (start + epoch_shift) % 100); > > ISTM that this is: (start + epoch_sh

Re: [PoC] Improve dead tuple storage for lazy vacuum

2021-07-08 Thread Andres Freund
Hi, On 2021-07-07 20:46:38 +0900, Masahiko Sawada wrote: > 1. Don't allocate more than 1GB. There was a discussion to eliminate > this limitation by using MemoryContextAllocHuge() but there were > concerns about point 2[1]. > > 2. Allocate the whole memory space at once. > > 3. Slow lookup perfor

Re: Skipping logical replication transactions on subscriber side

2021-07-08 Thread Alexey Lesovsky
On Fri, Jul 9, 2021 at 5:43 AM Masahiko Sawada wrote: > On Tue, Jul 6, 2021 at 7:13 PM Alexey Lesovsky wrote: > > > > On Tue, Jul 6, 2021 at 10:58 AM Masahiko Sawada > wrote: > >> > >> > Also, I'd like to suggest thinking twice about the view name (and > function used in view DDL) - "pg_stat_l

RE: Added schema level support for publication.

2021-07-08 Thread houzj.f...@fujitsu.com
On Thursday, July 8, 2021 11:47 AM houzj.f...@fujitsu.com wrote > On Wednesday, June 30, 2021 7:43 PM vignesh C > wrote: > > Thanks for reporting this issue, the attached v9 patch fixes this issue. > > This also > fixes the other issue you reported at [1]. > > Hi, > > I had a look at the patc

Re: bugfix: when the blocksize is 32k, the function page_header of pageinspect returns negative numbers.

2021-07-08 Thread Quan Zongliang
On 2021/7/9 9:50 上午, Michael Paquier wrote: On Fri, Jul 09, 2021 at 09:26:37AM +0800, Quan Zongliang wrote: new patch attached That's mostly fine at quick glance. Here are some comments. Please add pageinspect--1.9--1.10.sql within the patch. Using git, you can do that with a simple "git

Re: ECPG bug fix: DECALRE STATEMENT and DEALLOCATE, DESCRIBE

2021-07-08 Thread Michael Paquier
On Thu, Jul 08, 2021 at 11:42:14AM +, kuroda.hay...@fujitsu.com wrote: > I already said above, I think that DEALLOCATE statement should > follow the linked connection, but I cannot decide about DESCRIBE. > I want to ask how do you think. I am not completely sure. It would be good to hear from

Re: Why ALTER SUBSCRIPTION ... SET (slot_name='none') requires subscription disabled?

2021-07-08 Thread Japin Li
On Thu, 08 Jul 2021 at 18:17, Amit Kapila wrote: > On Thu, Jul 8, 2021 at 3:43 PM Japin Li wrote: >> >> On Thu, 08 Jul 2021 at 17:51, Amit Kapila wrote: >> > On Wed, Jul 7, 2021 at 7:25 PM Japin Li wrote: >> >> >> >> Hi, hackers >> >> >> >> The documentation [1] says: >> >> >> >> When dropping

Re: Teach pg_receivewal to use lz4 compression

2021-07-08 Thread Michael Paquier
On Thu, Jul 08, 2021 at 02:18:40PM +, gkokola...@pm.me wrote: > please find v2 of the patch which tries to address the commends > received so far. Thanks! > Michael Paquier wrote: >> + system_or_bail('lz4', '-t', $lz4_wals[0]); >> I think that you should just drop this part of the test. Th

Re: Inaccurate error message when set fdw batch_size to 0

2021-07-08 Thread Bharath Rupireddy
On Fri, Jul 9, 2021 at 6:25 AM Fujii Masao wrote: > The patch could not be applied cleanly because of recent commit d854720df6. > Could you rebase the patch? Thanks. Done. > - /* these must have a non-negative numeric value */ > + /* these must have a

Re: Grammar railroad diagram

2021-07-08 Thread Andres Freund
Hi, On 2021-07-03 10:39:02 +0200, Domingo Alvarez Duarte wrote: > I've done a experimental tool to convert bison grammars to a kind of EBNF > understood by https://www.bottlecaps.de/rr/ui It'd be nice if you could share that tool. The diagrams this can generate are neat... Greetings, Andres Fre

Re: Overflow hazard in pgbench

2021-07-08 Thread Andres Freund
Hi, On 2021-06-27 16:21:46 -0400, Tom Lane wrote: > BTW, for grins I tried building today's HEAD without -fwrapv, using > gcc version 11.1.1 20210531 (Red Hat 11.1.1-3) (GCC) > which is the newest version I have at hand. Not very surprisingly, > that reproduced the failure shown on moonjell

Re: ERROR: "ft1" is of the wrong type.

2021-07-08 Thread Michael Paquier
On Fri, Jul 09, 2021 at 10:44:13AM +0900, Kyotaro Horiguchi wrote: > Sounds reasonable. So the attached are that for PG11-PG14. 11 and 12 > shares the same patch. How much do the regression tests published upthread in https://postgr.es/m/20210219.173039.609314751334535042.horikyota@gmail.com

Re: bugfix: when the blocksize is 32k, the function page_header of pageinspect returns negative numbers.

2021-07-08 Thread Michael Paquier
On Fri, Jul 09, 2021 at 09:26:37AM +0800, Quan Zongliang wrote: > new patch attached That's mostly fine at quick glance. Here are some comments. Please add pageinspect--1.9--1.10.sql within the patch. Using git, you can do that with a simple "git add". With the current shape of the patch, one

Re: ERROR: "ft1" is of the wrong type.

2021-07-08 Thread Kyotaro Horiguchi
At Thu, 8 Jul 2021 10:02:53 +0200, Peter Eisentraut wrote in > My patch is now committed. The issue that started this thread now behaves > like this: > > ALTER TABLE ft1 ATTACH PARTITION ...; > ERROR: ALTER action ATTACH PARTITION cannot be performed on relation "ft1" > DETAIL: This operatio

Re: Incorrect usage of strtol, atoi for non-numeric junk inputs

2021-07-08 Thread Michael Paquier
On Thu, Jul 08, 2021 at 05:30:23PM +0900, Kyotaro Horiguchi wrote: > Looked through the three threads. Thanks! > [1] is trying to expose pg_strtoint16/32 to frontend, but I don't see > much point in doing that in conjunction with [2] or this thread. Since > the integral parameter values of pg-com

Re: bugfix: when the blocksize is 32k, the function page_header of pageinspect returns negative numbers.

2021-07-08 Thread Quan Zongliang
On 2021/7/8 3:54 下午, Michael Paquier wrote: On Wed, Jul 07, 2021 at 11:28:06PM -0500, Justin Pryzby wrote: I think you can refer to this prior commit for guidance. commit f18aa1b203930ed28cfe42e82d3418ae6277576d Author: Peter Eisentraut Date: Tue Jan 19 10:28:05 2021 +0100 pageinspec

Re: Outdated replication protocol error?

2021-07-08 Thread Andres Freund
Hi, On 2021-06-17 18:13:57 -0700, Jeff Davis wrote: > On Wed, 2021-06-16 at 16:17 -0700, Andres Freund wrote: > > I think we should explicitly compute the current timeline before > > using > > ThisTimelineID. E.g. in StartReplication() call a new version of > > GetFlushRecPtr() that also returns t

Re: Inaccurate error message when set fdw batch_size to 0

2021-07-08 Thread Fujii Masao
On 2021/05/26 15:22, Bharath Rupireddy wrote: On Thu, May 20, 2021 at 2:43 PM Bharath Rupireddy wrote: Thanks. That looks better. PSA v4 patch. Attaching v5 patch rebased on latest master. The patch could not be applied cleanly because of recent commit d854720df6. Could you rebase the p

Re: Skipping logical replication transactions on subscriber side

2021-07-08 Thread Masahiko Sawada
On Tue, Jul 6, 2021 at 7:13 PM Alexey Lesovsky wrote: > > On Tue, Jul 6, 2021 at 10:58 AM Masahiko Sawada wrote: >> >> > Also, I'd like to suggest thinking twice about the view name (and function >> > used in view DDL) - "pg_stat_logical_replication_error" contains very >> > common "logical re

Re: Skipping logical replication transactions on subscriber side

2021-07-08 Thread Masahiko Sawada
On Thu, Jul 8, 2021 at 6:28 PM Amit Kapila wrote: > > On Wed, Jul 7, 2021 at 11:47 AM Masahiko Sawada wrote: > > > > On Tue, Jul 6, 2021 at 6:33 PM Amit Kapila wrote: > > > > > > > > > > > According to the doc, ALTER SUBSCRIPTION ... SET is used to alter > > > > parameters originally set by CREA

Re: PROXY protocol support

2021-07-08 Thread Jacob Champion
Hi Magnus, I'm only just starting to page this back into my head, so this is by no means a full review of the v7 changes -- just stuff I've noticed over the last day or so of poking around. On Tue, 2021-06-29 at 11:48 +0200, Magnus Hagander wrote: > On Thu, Mar 11, 2021 at 12:05 AM Jacob Champion

Re: [PoC] Improve dead tuple storage for lazy vacuum

2021-07-08 Thread Hannu Krosing
On Fri, Jul 9, 2021 at 12:34 AM Peter Geoghegan wrote: > ... > > I would say that 200 TIDs per leaf page is common and ~1350 TIDs per > leaf page is not uncommon (with deduplication). Seems like that might > be enough? Likely yes, and also it would have the nice property of not changing the index

Re: [HACKERS] logical decoding of two-phase transactions

2021-07-08 Thread Peter Smith
On Thu, Jul 8, 2021 at 10:08 PM vignesh C wrote: > > On Thu, Jul 8, 2021 at 11:37 AM Amit Kapila wrote: > > > > On Tue, Jul 6, 2021 at 9:58 AM Peter Smith wrote: > > > > > > Please find attached the latest patch set v93* > > > > > > > Thanks, I have gone through the 0001 patch and made a number

Re: [PoC] Improve dead tuple storage for lazy vacuum

2021-07-08 Thread Peter Geoghegan
On Thu, Jul 8, 2021 at 1:53 PM Hannu Krosing wrote: > How I am approaching this is separating "page search" tyo run over a > (naturally) sorted array of 32 bit page pointers and only when the > page is found the indexes in this array are used to look up the > in-page bitmaps. > This allows the hea

Re: [PoC] Improve dead tuple storage for lazy vacuum

2021-07-08 Thread Peter Geoghegan
On Thu, Jul 8, 2021 at 1:47 AM Masahiko Sawada wrote: > As I wrote in the first email, I think there are two important factors > in index vacuuming performance: the performance to check if heap TID > that an index tuple points to is dead, and the number of times to > perform index bulk-deletion. T

Re: "debug_invalidate_system_caches_always" is too long

2021-07-08 Thread Alvaro Herrera
On 2021-Jul-08, Tom Lane wrote: > Robert Haas writes: > > On Wed, Jul 7, 2021 at 11:17 AM Tom Lane wrote: > >> Fair point. What do you think of the alternative proposals > >> "debug_flush_caches", "debug_discard_caches", etc? > > > I like debug_discard_caches best. > > I can live with that.

Re: [PoC] Improve dead tuple storage for lazy vacuum

2021-07-08 Thread Hannu Krosing
Resending as forgot to send to the list (thanks Peter :) ) On Wed, Jul 7, 2021 at 10:24 PM Peter Geoghegan wrote: > > The loop inside btvacuumpage() makes each loop iteration call the > callback -- this is always a call to lazy_tid_reaped() in practice. > And that's where we do binary searches. T

Re: "debug_invalidate_system_caches_always" is too long

2021-07-08 Thread Tom Lane
Robert Haas writes: > On Wed, Jul 7, 2021 at 11:17 AM Tom Lane wrote: >> Fair point. What do you think of the alternative proposals >> "debug_flush_caches", "debug_discard_caches", etc? > I like debug_discard_caches best. I can live with that. Anyone strongly against it?

Re: More time spending with "delete pending"

2021-07-08 Thread Alexander Lakhin
08.07.2021 10:47, Michael Paquier wrote: > On Thu, Jul 08, 2021 at 07:00:01AM +0300, Alexander Lakhin wrote: >> As Tom Lane noted above, the code added with bed90759f is dubious >> (_NtQueryInformationFile() can not be used to handle the "delete >> pending" state as CreateFile() returns INVALID_HAN

Re: Pipeline mode and PQpipelineSync()

2021-07-08 Thread Boris Kolpackov
Alvaro Herrera writes: > To be honest, I am hesitant to changing the charter in that way; I fear > it may have consequences I don't foresee. I think the workaround is not > *that* bad. Ok, fair enough. I've updated my code to account for this and it seems to be working fine now. > I'm having

Re: Replace remaining castNode(…, lfirst(…)) and friends calls with l*_node()

2021-07-08 Thread Tom Lane
Alvaro Herrera writes: > On 2021-Jul-07, Dagfinn Ilmari Mannsåker wrote: >> PartitionRangeDatum *datum = >> -castNode(PartitionRangeDatum, lfirst(cell)); >> +lfirst_node(PartitionRangeDatum, cell); > This is pretty personal and subjective, but stylistically I dislike > ini

Re: Replace remaining castNode(…, lfirst(…)) and friends calls with l*_node()

2021-07-08 Thread Alvaro Herrera
+1 on the idea. On a quick glance, all changes looks good. On 2021-Jul-07, Dagfinn Ilmari Mannsåker wrote: > diff --git a/src/backend/rewrite/rewriteSearchCycle.c > b/src/backend/rewrite/rewriteSearchCycle.c > index 599fe8e735..c50ebdba24 100644 > --- a/src/backend/rewrite/rewriteSearchCycle.c

Re: Pipeline mode and PQpipelineSync()

2021-07-08 Thread Alvaro Herrera
Looking at this again, I noticed that I could probably do away with the switch on pipelineStatus, and just call pqPipelineProcessQueue in all cases when appending commands to the queue; I *think* that will do the right thing in all cases. *Except* that I don't know what will happen if the program

Re: enable_resultcache confusion

2021-07-08 Thread David G. Johnston
On Thu, Jul 8, 2021 at 11:00 AM Tom Lane wrote: > Maybe name the plan node type Memoize, and the GUC "enable_memoize"? > > +1 David J.

Re: enable_resultcache confusion

2021-07-08 Thread Justin Pryzby
On Thu, Jul 08, 2021 at 12:51:45PM -0400, Bruce Momjian wrote: > Are we going to be forever explaining that enable_resultcache doesn't > cache query results? Do we need a different name? > enable_innerjoin_cache? See also https://www.postgresql.org/message-id/caaphdvos7z90hyiux3kcfe0q_4wywfwfab

Re: enable_resultcache confusion

2021-07-08 Thread Daniel Gustafsson
> On 8 Jul 2021, at 19:52, David G. Johnston wrote: > "enable_nodememoization" would work for me Include "node" concatenated with other words risks users reading it as "enable no demomoization", with confusion following. -- Daniel Gustafsson https://vmware.com/

Re: enable_resultcache confusion

2021-07-08 Thread Tom Lane
"David G. Johnston" writes: > "enable_nodememoization" would work for me That seems pretty unreadable. Maybe just "enable_memoization"? Really if we're going to do something here, we can't merely mess with the GUC name. David had expressed a willingness to rename everything about ResultCache s

Re: enable_resultcache confusion

2021-07-08 Thread David G. Johnston
On Thu, Jul 8, 2021 at 10:29 AM Robert Haas wrote: > On Thu, Jul 8, 2021 at 12:51 PM Bruce Momjian wrote: > > Are we going to be forever explaining that enable_resultcache doesn't > > cache query results? > > Yes, I can see that causing ongoing confusion. Naming things is really > hard... > > I

Re: [PATCH] expand the units that pg_size_pretty supports on output

2021-07-08 Thread Dean Rasheed
On Thu, 8 Jul 2021 at 14:38, David Rowley wrote: > > I gave it a bit of exercise by running pgbench and calling this procedure: > > It ran 8526956 times, so with the loop that's 8.5 billion random > numbers. No variations between the two functions. I got the same > after removing the 0 - to test

Re: "debug_invalidate_system_caches_always" is too long

2021-07-08 Thread Robert Haas
On Wed, Jul 7, 2021 at 11:17 AM Tom Lane wrote: > Peter Eisentraut writes: > > The clobbering doesn't actually happen unless you turn on > > CLOBBER_FREED_MEMORY, so it would be good to keep that separate. > > Fair point. What do you think of the alternative proposals > "debug_flush_caches", "de

Re: Pipeline mode and PQpipelineSync()

2021-07-08 Thread Alvaro Herrera
On 2021-Jul-08, Boris Kolpackov wrote: > Alvaro Herrera writes: > > That seems a fair complaint, but I think PQisBusy is doing the right > > thing per its charter. It is documented as "would PQgetResult block?" > > and it is returning correctly that PQgetResult would not block in that > > situa

Re: enable_resultcache confusion

2021-07-08 Thread Robert Haas
On Thu, Jul 8, 2021 at 12:51 PM Bruce Momjian wrote: > Are we going to be forever explaining that enable_resultcache doesn't > cache query results? Yes, I can see that causing ongoing confusion. Naming things is really hard... -- Robert Haas EDB: http://www.enterprisedb.com

Re: pgbench logging broken by time logic changes

2021-07-08 Thread Fabien COELHO
Hello Hannu, I'm not sure we have transaction lasts for very short time that nanoseconds matters. Nanoseconds may not matter yet, but they could be handy when for example we want to determine the order of parallel query executions. We are less than an order of magnitude away from being able

Re: Have I found an interval arithmetic bug?

2021-07-08 Thread Zhihong Yu
On Wed, Jun 30, 2021 at 9:35 AM Bruce Momjian wrote: > On Tue, Jun 29, 2021 at 06:49:45PM +0200, Daniel Gustafsson wrote: > > > On 29 Jun 2021, at 18:50, Zhihong Yu wrote: > > > > > Now that PG 15 is open for commit, do you think the patch can land ? > > > > Adding it to the commitfest patch tra

Re: pgbench logging broken by time logic changes

2021-07-08 Thread Fabien COELHO
Hello Thomas, Thanks! This doesn't seem to address the complaint, though. Don't you need to do something like this? (See also attached.) +initStats(&aggs, start - (start + epoch_shift) % 100); ISTM that this is: (start + epoch_shift) / 100 * 100 That should reproduce wha

enable_resultcache confusion

2021-07-08 Thread Bruce Momjian
Are we going to be forever explaining that enable_resultcache doesn't cache query results? Do we need a different name? enable_innerjoin_cache? -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com If only the physical world exists

Add statistics refresh materialized view

2021-07-08 Thread Seino Yuki
Hi. This is a proposal for a new feature in statistics collector. I think we need to add statistics about refresh matview to pg_stat_all_tables view. When the "REFRESH MATERIALIZED VIEW" was executed, the number of times it was executed and date it took were not recorded anywhere. "pg_stat_

Re: Replace remaining castNode(…, lfirst(…)) and friends calls with l*_node()

2021-07-08 Thread Dagfinn Ilmari Mannsåker
Daniel Gustafsson writes: >> On 7 Jul 2021, at 21:12, Dagfinn Ilmari Mannsåker wrote: > >> Here's a patch to convert the remaining ones. > > I haven't tested it yet, but +1 on the idea of cleaning these up making the > codebase consistent. FWIW, it passes `make check-world` on an assert- and TA

Re: pgsql: Don't try to print data type names in slot_store_error_callback(

2021-07-08 Thread Tom Lane
Robert Haas writes: > On Fri, Jul 2, 2021 at 4:05 PM Tom Lane wrote: >> The existing code tried to do syscache lookups in an already-failed >> transaction, which is problematic to say the least. > Why didn't the assertion in SearchCatCacheInternal() catch this? Was > this code never actually tes

Re: Pipeline mode and PQpipelineSync()

2021-07-08 Thread Boris Kolpackov
Alvaro Herrera writes: > On 2021-Jul-08, Boris Kolpackov wrote: > > > Alvaro Herrera writes: > > > > > Hmm ... aren't you trying to read more results than you sent queries? > > > > Hm, but should I be able to? Or, to put another way, should PQisBusy() > > indicate there is a result available

Re: pgsql: Don't try to print data type names in slot_store_error_callback(

2021-07-08 Thread Robert Haas
On Fri, Jul 2, 2021 at 4:05 PM Tom Lane wrote: > The existing code tried to do syscache lookups in an already-failed > transaction, which is problematic to say the least. Why didn't the assertion in SearchCatCacheInternal() catch this? Was this code never actually tested? -- Robert Haas EDB: ht

Re: pgbench logging broken by time logic changes

2021-07-08 Thread Hannu Krosing
On Thu, Jun 17, 2021 at 7:18 AM Kyotaro Horiguchi wrote: > > I'm not sure we have transaction lasts for very short time that > nanoseconds matters. > Nanoseconds may not matter yet, but they could be handy when for example we want to determine the order of parallel query executions. We are less

Re: Teach pg_receivewal to use lz4 compression

2021-07-08 Thread gkokolatos
Hi, please find v2 of the patch which tries to address the commends received so far. Thank you all for your comments. Michael Paquier wrote: > Documentation is missing from the patch. > It has now been added. > + LZ4F_compressionContext_t ctx; > + size_t outbufCapacity; > + void

Re: SHA-1 FIPS - compliance

2021-07-08 Thread Bruce Momjian
On Thu, Jul 8, 2021 at 02:33:33PM +0200, Filip Janus wrote: > Hi all, > I am a new maintainer of PostgreSQL in Fedora and RHEL. Currently, I am > solving > usage SHA-1  for key-derivation in pgcrypto (the s2k-digest-algo).  In the > documentation, I have found that there are options SHA-1 or MD5.

Re: Pipeline mode and PQpipelineSync()

2021-07-08 Thread Alvaro Herrera
On 2021-Jul-08, Boris Kolpackov wrote: > Alvaro Herrera writes: > > > Hmm ... aren't you trying to read more results than you sent queries? > > Hm, but should I be able to? Or, to put another way, should PQisBusy() > indicate there is a result available without me sending a query for it? > That

Re: visibility map corruption

2021-07-08 Thread Bruce Momjian
On Thu, Jul 8, 2021 at 08:11:14AM -0500, Justin Pryzby wrote: > Also, the pg_upgrade status message still seems to be misplaced: > > In 20210706190612.gm22...@telsasoft.com, Justin Pryzby wrote: > > I re-arranged the pg_upgrade output of that patch: it was in the middle of > > the > > two halves

Re: [PoC] Improve dead tuple storage for lazy vacuum

2021-07-08 Thread Hannu Krosing
Very nice results. I have been working on the same problem but a bit different solution - a mix of binary search for (sub)pages and 32-bit bitmaps for tid-in-page. Even with currebnt allocation heuristics (allocate 291 tids per page) it initially allocate much less space, instead of current 291*6

Re: Enhanced error message to include hint messages for redundant options error

2021-07-08 Thread vignesh C
On Thu, Jul 8, 2021 at 1:52 AM Daniel Gustafsson wrote: > > > On 6 Jul 2021, at 17:08, vignesh C wrote: > > > The patch was not applying on the head because of the recent commit > > "8aafb02616753f5c6c90bbc567636b73c0cbb9d4", attached patch which is > > rebased on HEAD. > > I sort of like the vis

Re: [PATCH] expand the units that pg_size_pretty supports on output

2021-07-08 Thread David Rowley
tOn Thu, 8 Jul 2021 at 20:23, Dean Rasheed wrote: > > > On Thu, 8 Jul 2021 at 13:31, David Rowley wrote: > > Here's a patch which I believe makes pg_size_pretty() and > > pg_size_pretty_numeric() match in regards to negative values. > > LGTM, except I think it's worth also making the numeric cod

Re: Small documentation improvement for ALTER SUBSCRIPTION

2021-07-08 Thread Daniel Gustafsson
> On 8 Jul 2021, at 15:00, Masahiko Sawada wrote: > I think we can improve it by changing to > 'refresh_option'. Thoughts? My first thought was that the existing wording is clearer, referring to “options to refresh”. But thinking on it more, it’s easy to see someone confusing the options part a

Re: visibility map corruption

2021-07-08 Thread Justin Pryzby
Also, the pg_upgrade status message still seems to be misplaced: In 20210706190612.gm22...@telsasoft.com, Justin Pryzby wrote: > I re-arranged the pg_upgrade output of that patch: it was in the middle of the > two halves: "Setting next transaction ID and epoch for new cluster" +++ b/src/bin/pg_up

Re: visibility map corruption

2021-07-08 Thread Bruce Momjian
On Thu, Jul 8, 2021 at 07:35:58AM +0200, Drouvot, Bertrand wrote: > Thanks for having looked at it. > > It looks good to me, but i have one question: > > +    printf(_("  -u, --oldest-transaction-id=XID  set oldest transaction > ID\n")); > > and > > +   if (!TransactionIdIsNorm

Small documentation improvement for ALTER SUBSCRIPTION

2021-07-08 Thread Masahiko Sawada
Hi all, When reading the doc of ALTER SUBSCRIPTION I realized that 'refresh options' in the following paragraph is not tagged: --- Additionally, refresh options as described under REFRESH PUBLICATION may be specified, except in the case of DROP PUBLICATION. --- When I read it for the first time,

Re: [PATCH] expand the units that pg_size_pretty supports on output

2021-07-08 Thread David Rowley
On Thu, 8 Jul 2021 at 07:31, Tom Lane wrote: > > David Christensen writes: > > Enclosed is the patch to change the return type to numeric, as well as one > > for expanding units to > > add PB and EB. > > Can we really get away with changing the return type? That would > by no stretch of the ima

SHA-1 FIPS - compliance

2021-07-08 Thread Filip Janus
Hi all, I am a new maintainer of PostgreSQL in Fedora and RHEL. Currently, I am solving usage SHA-1 for key-derivation in pgcrypto (the s2k-digest-algo). In the documentation , I have found that there are options SHA-1 or MD5. Unfortunately, none

Re: rand48 replacement

2021-07-08 Thread Fabien COELHO
Hello Yura, Given 99.99% cases will be in the likely case, branch predictor should eliminate decision cost. Hmmm. ISTM that a branch predictor should predict that unknown < small should probably be false, so a hint should be given that it is really true. Why? Branch predictor is history ba

Re: rand48 replacement

2021-07-08 Thread Fabien COELHO
Finally, I think it would be better to treat the upper bound of the range as inclusive. This bothered me as well, but the usual approach seems to use range as the number of values, so I was hesitant to depart from that. I'm still hesitant to go that way. Yeah, that bothered me too. For exam

Re: pgbench logging broken by time logic changes

2021-07-08 Thread Thomas Munro
On Thu, Jul 1, 2021 at 8:50 PM Fabien COELHO wrote: > Attached a v14 in that spirit. Thanks! This doesn't seem to address the complaint, though. Don't you need to do something like this? (See also attached.) +initStats(&aggs, start - (start + epoch_shift) % 100); That should reproduc

Re: [HACKERS] logical decoding of two-phase transactions

2021-07-08 Thread vignesh C
On Thu, Jul 8, 2021 at 11:37 AM Amit Kapila wrote: > > On Tue, Jul 6, 2021 at 9:58 AM Peter Smith wrote: > > > > Please find attached the latest patch set v93* > > > > Thanks, I have gone through the 0001 patch and made a number of > changes. (a) Removed some of the code which was leftover from p

RE: ECPG bug fix: DECALRE STATEMENT and DEALLOCATE, DESCRIBE

2021-07-08 Thread kuroda.hay...@fujitsu.com
Dear Michael, > I have been chewing on this comment and it took me some time to > understand what you meant here. Sorry... But your understanding is correct. > It is true that the ecpglib part, aka > all the routines you are quoting above, don't rely at all on the > connection names. However, t

Re: Diagnostic comment in LogicalIncreaseXminForSlot

2021-07-08 Thread Masahiko Sawada
On Thu, Jul 8, 2021 at 8:14 PM Daniel Gustafsson wrote: > > > On 8 Jul 2021, at 12:56, Amit Kapila wrote: > > > Do you or others have any opinion on whether this should be > > back-patched? I personally prefer it to be a HEAD-only patch. > > +1 for only applying this to HEAD. The restart_lsn deb

Re: A few assorted typos in comments

2021-07-08 Thread Daniel Gustafsson
> On 8 Jul 2021, at 09:37, Michael Paquier wrote: > > On Thu, Jul 08, 2021 at 10:19:00AM +1200, Thomas Munro wrote: >> LGTM. > > +1. Pushed (387925893e), thanks. -- Daniel Gustafsson https://vmware.com/

Re: Diagnostic comment in LogicalIncreaseXminForSlot

2021-07-08 Thread Daniel Gustafsson
> On 8 Jul 2021, at 12:56, Amit Kapila wrote: > Do you or others have any opinion on whether this should be > back-patched? I personally prefer it to be a HEAD-only patch. +1 for only applying this to HEAD. The restart_lsn debug elog has been there since 2014 so there doesn’t seem to be any imm

Re: Diagnostic comment in LogicalIncreaseXminForSlot

2021-07-08 Thread Amit Kapila
On Mon, Jul 5, 2021 at 12:54 PM Masahiko Sawada wrote: > > On Fri, May 21, 2021 at 6:00 PM Ashutosh Bapat > wrote: > > > > > > > > On Fri, May 21, 2021 at 11:26 AM Amit Kapila > > wrote: > >> > >> On Thu, May 20, 2021 at 5:43 PM Ashutosh Bapat > >> wrote: > >> > > >> > Hi > >> > LogicalIncreas

Re: Why ALTER SUBSCRIPTION ... SET (slot_name='none') requires subscription disabled?

2021-07-08 Thread Amit Kapila
On Thu, Jul 8, 2021 at 3:43 PM Japin Li wrote: > > On Thu, 08 Jul 2021 at 17:51, Amit Kapila wrote: > > On Wed, Jul 7, 2021 at 7:25 PM Japin Li wrote: > >> > >> Hi, hackers > >> > >> The documentation [1] says: > >> > >> When dropping a subscription that is associated with a replication slot on

Re: Why ALTER SUBSCRIPTION ... SET (slot_name='none') requires subscription disabled?

2021-07-08 Thread Japin Li
On Thu, 08 Jul 2021 at 17:51, Amit Kapila wrote: > On Wed, Jul 7, 2021 at 7:25 PM Japin Li wrote: >> >> Hi, hackers >> >> The documentation [1] says: >> >> When dropping a subscription that is associated with a replication slot on >> the >> remote host (the normal state), DROP SUBSCRIPTION wil

Re: bugfix: when the blocksize is 32k, the function page_header of pageinspect returns negative numbers.

2021-07-08 Thread Quan Zongliang
On 2021/7/8 3:54 下午, Michael Paquier wrote: On Wed, Jul 07, 2021 at 11:28:06PM -0500, Justin Pryzby wrote: I think you can refer to this prior commit for guidance. commit f18aa1b203930ed28cfe42e82d3418ae6277576d Author: Peter Eisentraut Date: Tue Jan 19 10:28:05 2021 +0100 pageinspe

Re: Why ALTER SUBSCRIPTION ... SET (slot_name='none') requires subscription disabled?

2021-07-08 Thread Amit Kapila
On Wed, Jul 7, 2021 at 7:25 PM Japin Li wrote: > > Hi, hackers > > The documentation [1] says: > > When dropping a subscription that is associated with a replication slot on the > remote host (the normal state), DROP SUBSCRIPTION will connect to the remote > host and try to drop the replication sl

Re: Skipping logical replication transactions on subscriber side

2021-07-08 Thread Amit Kapila
On Wed, Jul 7, 2021 at 11:47 AM Masahiko Sawada wrote: > > On Tue, Jul 6, 2021 at 6:33 PM Amit Kapila wrote: > > > > > > > > According to the doc, ALTER SUBSCRIPTION ... SET is used to alter > > > parameters originally set by CREATE SUBSCRIPTION. Therefore, we can > > > specify a subset of parame

Re: rand48 replacement

2021-07-08 Thread Dean Rasheed
On Thu, 8 Jul 2021 at 09:26, Fabien COELHO wrote: > > > Finally, I think it would be better to treat the upper bound of the > > range as inclusive. > > This bothered me as well, but the usual approach seems to use range as the > number of values, so I was hesitant to depart from that. I'm still >

Re: [PoC] Improve dead tuple storage for lazy vacuum

2021-07-08 Thread Masahiko Sawada
On Thu, Jul 8, 2021 at 5:24 AM Peter Geoghegan wrote: > > On Wed, Jul 7, 2021 at 4:47 AM Masahiko Sawada wrote: > > Currently, the TIDs of dead tuples are stored in an array that is > > collectively allocated at the start of lazy vacuum and TID lookup uses > > bsearch(). There are the following c

Re: Incorrect usage of strtol, atoi for non-numeric junk inputs

2021-07-08 Thread Kyotaro Horiguchi
At Wed, 7 Jul 2021 17:40:13 +0530, Bharath Rupireddy wrote in > On Fri, Jun 4, 2021 at 10:23 PM Alvaro Herrera > wrote: > > > > On 2021-Jun-04, Bharath Rupireddy wrote: > > > > > On Fri, Jun 4, 2021 at 8:58 PM Alvaro Herrera > > > wrote: > > > > > > I would suggest that the best way forward

Re: rand48 replacement

2021-07-08 Thread Fabien COELHO
Hello Dean, Whilst it has been interesting learning and discussing all these different techniques, I think it's probably best to stick with the bitmask method, rather than making the code too complex and difficult to follow. Yes. The bitmask method has the advantage of being very simple, eas

Re: [PATCH] expand the units that pg_size_pretty supports on output

2021-07-08 Thread Dean Rasheed
On Thu, 8 Jul 2021 at 05:30, David Rowley wrote: > > On Thu, 8 Jul 2021 at 13:31, David Rowley wrote: > > It feels like if we're going to fix this negative rounding thing then > > we should maybe do it and backpatch a fix then rebase this work on top > > of that. Yes, that was my thinking too.

Re: ERROR: "ft1" is of the wrong type.

2021-07-08 Thread Peter Eisentraut
On 02.07.21 06:20, Michael Paquier wrote: On Wed, Jun 30, 2021 at 01:43:52PM +0500, Ahsan Hadi wrote: Sure I will also take a look at this patch. +1 for avoiding the backpatching burden. From what I recall of this thread, nobody has really complained about this stuff either, so a backpatch

Re: bugfix: when the blocksize is 32k, the function page_header of pageinspect returns negative numbers.

2021-07-08 Thread Michael Paquier
On Wed, Jul 07, 2021 at 11:28:06PM -0500, Justin Pryzby wrote: > I think you can refer to this prior commit for guidance. > > commit f18aa1b203930ed28cfe42e82d3418ae6277576d > Author: Peter Eisentraut > Date: Tue Jan 19 10:28:05 2021 +0100 > > pageinspect: Change block number arguments to

Re: wrong relkind error messages

2021-07-08 Thread Peter Eisentraut
On 02.07.21 18:10, Alvaro Herrera wrote: On 2021-Jun-24, Peter Eisentraut wrote: There might be room for some wordsmithing in a few places, but generally I think this is complete. This looks good to me. I am +0.1 on your proposal of "cannot have triggers" vs Michael's "cannot create triggers

Re: More time spending with "delete pending"

2021-07-08 Thread Michael Paquier
On Thu, Jul 08, 2021 at 07:00:01AM +0300, Alexander Lakhin wrote: > As Tom Lane noted above, the code added with bed90759f is dubious > (_NtQueryInformationFile() can not be used to handle the "delete > pending" state as CreateFile() returns INVALID_HANDLE_VALUE in this case.) > Probably that chang

  1   2   >