Re: [HACKERS] Custom compression methods

2021-03-15 Thread Andres Freund
Hi, On 2021-03-15 15:29:05 -0400, Robert Haas wrote: > On Mon, Mar 15, 2021 at 8:14 AM Dilip Kumar wrote: > > In the attached patches I have changed this, ... > > OK, so just looking over this patch series, here's what I think: > > - 0001 and 0002 are now somewhat independent of the rest of this

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-03-15 Thread Alvaro Herrera
On 2021-Feb-26, Alvaro Herrera wrote: > Hmm, but if we take this approach, then we're still vulnerable to the > problem that somebody can do DETACH CONCURRENTLY and cancel the wait (or > crash the server), then mess up the state before doing DETACH FINALIZE: > when they cancel the wait, the lock w

Re: New IndexAM API controlling index vacuum strategies

2021-03-15 Thread Andres Freund
Hi, On 2021-03-15 12:58:33 -0700, Peter Geoghegan wrote: > On Mon, Mar 15, 2021 at 12:21 PM Andres Freund wrote: > > It's evil sorcery. Fragile sorcery. I think Robert, Tom and me all run > > afoul of edge cases around it in the last few years. > > Right, which is why I thought that I might be m

Re: fdatasync performance problem with large number of DB files

2021-03-15 Thread Thomas Munro
On Tue, Mar 16, 2021 at 3:30 AM Paul Guo wrote: > By the way, there is a usual case that we could skip fsync: A fsync-ed > already standby generated by pg_rewind/pg_basebackup. > The state of those standbys are surely not > DB_SHUTDOWNED/DB_SHUTDOWNED_IN_RECOVERY, so the > pgdata directory is fs

Re: New IndexAM API controlling index vacuum strategies

2021-03-15 Thread Andres Freund
Hi, On 2021-03-15 13:58:02 -0700, Peter Geoghegan wrote: > On Mon, Mar 15, 2021 at 12:58 PM Peter Geoghegan wrote: > > > I'm not comfortable with this change without adding more safety > > > checks. If there's ever a case in which the HEAPTUPLE_DEAD case is hit > > > and the xid needs to be froze

make the stats collector shutdown without writing the statsfiles if the immediate shutdown is requested.

2021-03-15 Thread Masahiro Ikeda
Hi, This thread came from another thread about collecting the WAL stats([1]). Is it better to make the stats collector shutdown without writing the stats files if the immediate shutdown is requested? There was a related discussion([2]) although it's stopped on 12/1/2016. IIUC, the thread's

Re: New IndexAM API controlling index vacuum strategies

2021-03-15 Thread Peter Geoghegan
On Mon, Mar 15, 2021 at 4:11 PM Andres Freund wrote: > > > I'm not comfortable with this change without adding more safety > > > checks. If there's ever a case in which the HEAPTUPLE_DEAD case is hit > > > and the xid needs to be frozen, we'll either cause errors or > > > corruption. Yes, that's a

Re: [PATCH]: Allow errors in parameter values to be reported during the BIND phase itself..

2021-03-15 Thread Tom Lane
Justin Pryzby writes: > On Mon, Mar 15, 2021 at 06:45:49PM -0400, Tom Lane wrote: >> I started to look at this, and immediately began to wonder where is the >> previous discussion you're evidently referring to. Can you dig up an >> archives link? > I think I was referring to this (from the commi

Re: Enhance traceability of wal_level changes for backup management

2021-03-15 Thread Laurenz Albe
On Mon, 2021-03-15 at 17:32 -0400, Stephen Frost wrote: > We explicitly document that people can switch the WAL level and restart > to do bulk data loads faster, and there's certainly no shortage of > discussion (including what prompted this thread..) about doing exactly > that. Adding more docume

Re: Improvements and additions to COPY progress reporting

2021-03-15 Thread Michael Paquier
On Mon, Mar 15, 2021 at 12:43:40PM +0100, Matthias van de Meent wrote: > Hmm, does CFBot not run checkout on windows with crlf line endings? I > had expected it to do as such. This is environment-sensitive, so I am not surprised that Appveyor changes the way newlines are handled there. I could se

Re: [PATCH]: Allow errors in parameter values to be reported during the BIND phase itself..

2021-03-15 Thread Justin Pryzby
On Mon, Mar 15, 2021 at 08:30:18PM -0400, Tom Lane wrote: > + /* Pop the error callback */ > + error_context_stack = error_context_stack->previous; > + > /* > * Once all parameters have been received, prepare for > printing them >

Re: Consider Parallelism While Planning For REFRESH MATERIALIZED VIEW

2021-03-15 Thread Thomas Munro
On Mon, Mar 15, 2021 at 8:25 PM Bharath Rupireddy wrote: > > > The problem with a case like REFRESH MATERIALIZED VIEW is that there's > > > nothing to prevent something that gets run in the course of the query > > > from trying to access the view (and the heavyweight lock won't prevent > > > that,

Re: shared-memory based stats collector

2021-03-15 Thread Andres Freund
Hi, On 2021-03-13 10:05:21 -0800, Andres Freund wrote: > Cool. I'll give it a try. I have a few questions about the patch: - Why was collect_oids() changed to a different hashtable as part of this change? Seems fairly independent? - What's the point of all those cached_* stuff? There's not a

Re: pg_amcheck contrib application

2021-03-15 Thread Mark Dilger
> On Mar 15, 2021, at 11:57 AM, Tom Lane wrote: > > Not sure that I believe the theory that this is from bad luck of > concurrent autovacuum timing, though. The fact that we're seeing > this on just those two animals suggests strongly to me that it's > architecture-correlated, instead. I find

Re: Type of wait events WalReceiverWaitStart and WalSenderWaitForWAL

2021-03-15 Thread Kyotaro Horiguchi
At Tue, 16 Mar 2021 03:12:54 +0900, Fujii Masao wrote in > The wait event WalReceiverWaitStart has been categorized in the type > Client. > But why? Walreceiver is waiting for startup process to set the lsn and > timeline while it is reporting WalReceiverWaitStart. So its type > should be IPC, >

RE: libpq debug log

2021-03-15 Thread tsunakawa.ta...@fujitsu.com
I'm looking at the last file libpq-trace.c. I'll continue the review after lunch. Below are some comments so far. (1) - Enables tracing of the client/server communication to a debugging file stream. + Enables tracing of the client/server communication to a debugging file + str

Re: subscriptionCheck failures

2021-03-15 Thread Amit Kapila
On Mon, Mar 15, 2021 at 6:00 PM Thomas Munro wrote: > > Hi, > > This seems to be a new low frequency failure, I didn't see it mentioned > already: > Thanks for reporting, I'll look into it. -- With Regards, Amit Kapila.

RE: subscriptionCheck failures

2021-03-15 Thread osumi.takami...@fujitsu.com
Hello On Tuesday, March 16, 2021 12:31 PM Amit Kapila wrote: > On Mon, Mar 15, 2021 at 6:00 PM Thomas Munro > wrote: > > > > Hi, > > > > This seems to be a new low frequency failure, I didn't see it mentioned > already: Oh, this is the test I wrote and included as part of the commit ce0fdbfe #

RE: make the stats collector shutdown without writing the statsfiles if the immediate shutdown is requested.

2021-03-15 Thread kuroda.hay...@fujitsu.com
Dear Ikeda-san I think the idea is good. I read the patch and other sources, and I found process_startup_packet_die also execute _exit(1). I think they can be combined into one function and moved to interrupt.c, but some important comments might be removed. How do you think? Best Regards, Hay

Re: Allow batched insert during cross-partition updates

2021-03-15 Thread Amit Langote
Hi Georgios, On Fri, Mar 12, 2021 at 7:59 PM wrote: > On Friday, March 12, 2021 3:45 AM, Amit Langote > wrote: > > On Thu, Mar 11, 2021 at 8:36 PM gkokola...@pm.me wrote: > > > On Thursday, March 11, 2021 9:42 AM, Amit Langote amitlangot...@gmail.com > > > wrote: > > > > What we do support how

Re: [HACKERS] Custom compression methods

2021-03-15 Thread Dilip Kumar
On Tue, Mar 16, 2021 at 4:28 AM Andres Freund wrote: Replying to some of the comments.. > - Is nodeModifyTable.c really the right place for the logic around > CompareCompressionMethodAndDecompress()? And is doing it in every > place that does "user initiated" inserts really the right way? W

Re: [HACKERS] Custom compression methods

2021-03-15 Thread Justin Pryzby
I'm a minor contributor now to a couple bits of this patch set, but I can answer a couple of these points. On Mon, Mar 15, 2021 at 03:58:35PM -0700, Andres Freund wrote: > Comments about 0003: > - why is HIDE_TOAST_COMPRESSION useful? Doesn't quite seem to be > comparable to HIDE_TABLEAM? That

Re: [HACKERS] Custom compression methods

2021-03-15 Thread Dilip Kumar
On Tue, Mar 16, 2021 at 11:18 AM Justin Pryzby wrote: > > I'm a minor contributor now to a couple bits of this patch set, but I can > answer a couple of these points. > > On Mon, Mar 15, 2021 at 03:58:35PM -0700, Andres Freund wrote: > > Comments about 0003: > > - why is HIDE_TOAST_COMPRESSION use

Re: pg_amcheck contrib application

2021-03-15 Thread Noah Misch
On Mon, Mar 15, 2021 at 02:57:20PM -0400, Tom Lane wrote: > Mark Dilger writes: > > On Mar 15, 2021, at 10:04 AM, Tom Lane wrote: > >> These animals have somewhat weird alignment properties: MAXALIGN is 8 > >> but ALIGNOF_DOUBLE is only 4. I speculate that that is affecting their > >> choices ab

RE: libpq debug log

2021-03-15 Thread tsunakawa.ta...@fujitsu.com
I've not finished reviewing yet, but there seems to be many mistakes. I'm sending second set of review comments now so you can fix them in parallel. (8) + charid = '\0'; This initialization is not required because id will always be assigned a value shortly. (9) +static int

Re: Type of wait events WalReceiverWaitStart and WalSenderWaitForWAL

2021-03-15 Thread Fujii Masao
On 2021/03/16 11:59, Kyotaro Horiguchi wrote: At Tue, 16 Mar 2021 03:12:54 +0900, Fujii Masao wrote in The wait event WalReceiverWaitStart has been categorized in the type Client. But why? Walreceiver is waiting for startup process to set the lsn and timeline while it is reporting WalReceiv

<    1   2