Re: Printing LSN made easy

2021-01-19 Thread Ashutosh Bapat
On Wed, Jan 20, 2021 at 11:55 AM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 2020-11-27 11:40, Ashutosh Bapat wrote: > > The solution seems to be simple though. In the attached patch, I have > > added two macros > > #define LSN_FORMAT "%X/%X" > > #define LSN_FORMAT_ARG(lsn) (u

Re: Discarding DISCARD ALL

2021-01-19 Thread Peter Eisentraut
On 2020-12-23 15:33, Simon Riggs wrote: Poolers such as pgbouncer would then be able to connect transaction mode pools by setting transaction_cleanup=on at time of connection, avoiding any need to issue a server_reset_query, removing the DISCARD ALL command from the normal execution path, while s

Re: Printing LSN made easy

2021-01-19 Thread Michael Paquier
On Wed, Jan 20, 2021 at 07:25:37AM +0100, Peter Eisentraut wrote: > It looks like we are not getting any consensus on this approach. One > reduced version I would consider is just the second part, so you'd write > something like > > snprintf(lsnchar, sizeof(lsnchar), "%X/%X", > L

Re: Release SPI plans for referential integrity with DISCARD ALL

2021-01-19 Thread Peter Eisentraut
On 2021-01-13 09:47, yuzuko wrote: But we are also considering another option to solve this problem, which makes users to release cached SPI plans for referential integrity as well as plain cached plans with DISCARD ALL. To do that, we added a new function, RI_DropAllPreparedPlan() which deletes

Re: a misbehavior of partition row movement (?)

2021-01-19 Thread Peter Eisentraut
On 2021-01-08 09:54, Amit Langote wrote: I don't quite recall if the decision to implement it like this was based on assuming that this is what users would like to see happen in this case or the perceived difficulty of implementing it the other way around, that is, of firing AFTER UPDATE triggers

Re: Boundary value check in lazy_tid_reaped()

2021-01-19 Thread Peter Eisentraut
On 2020-10-30 02:43, Masahiko Sawada wrote: Using the integer set is very memory efficient (5MB vs. 114MB in the base case) and there is no 1GB limitation. Looking at the execution time, I had expected that using the integer set is slower on recording TIDs than using the simple array but in this

Re: list of extended statistics on psql

2021-01-19 Thread Tatsuro Yamada
Hi Tomas, On 2021/01/20 11:35, Tatsuro Yamada wrote: Apologies for all the extra work - I haven't realized this flaw when pushing for showing more stuff :-( Don't worry about it. We didn't notice the problem even when viewed by multiple people on -hackers. Let's keep moving forward. :-D I'll

Re: New IndexAM API controlling index vacuum strategies

2021-01-19 Thread Masahiko Sawada
On Wed, Jan 20, 2021 at 9:45 AM Peter Geoghegan wrote: > > On Tue, Jan 19, 2021 at 2:57 PM Peter Geoghegan wrote: > > * Maybe it would be better if you just changed the definition such > > that "MAXALIGN(SizeofHeapTupleHeader)" became "MAXIMUM_ALIGNOF", with > > no other changes? (Some variant of

Re: [PATCH 1/1] Initial mach based shared memory support.

2021-01-19 Thread Tom Lane
Peter Eisentraut writes: > This is the first I've heard in years of shared memory limits being a > problem on macOS. What settings or what levels of concurrency do you > use to provoke these errors? I suppose it wouldn't be too hard to run into shmmni with aggressive parallel testing; the defa

Re: Odd, intermittent failure in contrib/pageinspect

2021-01-19 Thread Michael Paquier
On Tue, Jan 19, 2021 at 05:03:49PM -0500, Tom Lane wrote: > It looks to me like heap_surgery ought to be okay, because it's operating > on a temp table; if there are any page access conflicts on that, we've > got BIG trouble ;-) Bah, of course. I managed to miss this part. > Poking around, I fou

Re: Printing LSN made easy

2021-01-19 Thread Peter Eisentraut
On 2020-11-27 11:40, Ashutosh Bapat wrote: The solution seems to be simple though. In the attached patch, I have added two macros #define LSN_FORMAT "%X/%X" #define LSN_FORMAT_ARG(lsn) (uint32) ((lsn) >> 32), (uint32) (lsn) which can be used instead. It looks like we are not getting any consen

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-19 Thread Fujii Masao
On 2021/01/19 12:09, Bharath Rupireddy wrote: On Mon, Jan 18, 2021 at 9:11 PM Fujii Masao wrote: Attaching v12 patch set. 0001 is for postgres_fdw_disconnect() function, 0002 is for keep_connections GUC and 0003 is for keep_connection server level option. Thanks! Please review it furthe

Re: [HACKERS] Custom compression methods

2021-01-19 Thread Dilip Kumar
On Wed, Jan 20, 2021 at 12:37 AM Justin Pryzby wrote: > > Thanks for updating the patch. Thanks for the review > On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby wrote: > > The most recent patch doesn't compile --without-lz4: > On Tue, Jan 05, 2021 at 11:19:33AM +0530, Dilip Kumar wrote: > > On Mon

Re: [PATCH 1/1] Initial mach based shared memory support.

2021-01-19 Thread Peter Eisentraut
On 2020-11-22 09:19, James Hilliard wrote: OSX implements sysv shmem support via a mach wrapper, however the mach sysv shmem wrapper has some severe restrictions that prevent us from allocating enough memory segments in some cases. These limits appear to be due to the way the wrapper itself is i

Re: pg_class.reltype -> pg_type.oid missing for pg_toast table

2021-01-19 Thread Joel Jacobson
On Tue, Jan 19, 2021, at 17:43, Tom Lane wrote: >I'm too lazy to check the code right now, but my recollection is that we >do not bother to make composite-type entries for toast tables. However, >they should have reltype = 0 if so, so I'm not quite sure where the >above failure is coming from. My

Re: Deleting older versions in unique indexes to avoid page splits

2021-01-19 Thread Peter Geoghegan
On Tue, Jan 19, 2021 at 7:54 PM Amit Kapila wrote: > The worst cases could be (a) when there is just one such duplicate > (indexval logically unchanged) on the page and that happens to be the > last item and others are new insertions, (b) same as (a) and along > with it lets say there is an open t

Re: Deleting older versions in unique indexes to avoid page splits

2021-01-19 Thread Andres Freund
Hi, On 2021-01-20 09:24:35 +0530, Amit Kapila wrote: > I feel extending the deletion mechanism based on the number of LP_DEAD > items sounds more favorable than giving preference to duplicate > items. Sure, it will give equally good or better results if there are > no long-standing open transactio

Re: Some coverage for DROP OWNED BY with pg_default_acl

2021-01-19 Thread Michael Paquier
On Tue, Jan 19, 2021 at 05:49:03PM -0300, Alvaro Herrera wrote: > Heh, interesting case. Added coverage is good, so +1. Thanks. I read through it again and applied the test. > Since the role regress_priv_user2 is "private" to the privileges.sql > script, there's no danger of a concurrent test g

Re: Stronger safeguard for archive recovery not to miss data

2021-01-19 Thread Fujii Masao
On 2021/01/20 1:05, Laurenz Albe wrote: On Mon, 2021-01-18 at 07:34 +, osumi.takami...@fujitsu.com wrote: I noticed that this message should cover both archive recovery modes, which means in recovery mode and standby mode. Then, I combined your suggestion above with this point of view. Ha

Re: Deleting older versions in unique indexes to avoid page splits

2021-01-19 Thread Amit Kapila
On Tue, Jan 19, 2021 at 3:03 AM Peter Geoghegan wrote: > > On Mon, Jan 18, 2021 at 1:10 PM Victor Yegorov wrote: > > I must admit, that it's a bit difficult to understand you here (at least > > for me). > > > > I assume that by "bet" you mean flagged tuple, that we marked as such > > (should we

Re: Add statistics to pg_stat_wal view for wal related parameter tuning

2021-01-19 Thread Masahiro Ikeda
On 2020-12-22 11:16, Masahiro Ikeda wrote: Thanks for your comments. On 2020-12-22 09:39, Andres Freund wrote: Hi, On 2020-12-21 13:16:50 -0800, Andres Freund wrote: On 2020-12-02 13:52:43 +0900, Fujii Masao wrote: > Pushed. Thanks! Why are wal_records/fpi long, instead of uint64? lo

Re: Change default of checkpoint_completion_target

2021-01-19 Thread japin
On Wed, 20 Jan 2021 at 03:47, Stephen Frost wrote: > Greetings, > > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> Stephen Frost writes: >> > Any further comments or thoughts on this one? >> >> This: >> >> +total time between checkpoints. The default is 0.9, which spreads >> the >> +

Re: list of extended statistics on psql

2021-01-19 Thread Tatsuro Yamada
Hi Tomas, On 2021/01/19 11:52, Tomas Vondra wrote: I'm going to create the WIP patch to use the above queriy. Any comments welcome. :-D Yes, I think using this simpler query makes sense. If we decide we need something more elaborate, we can improve that by in future PostgreSQL versions (af

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-01-19 Thread Julien Rouhaud
Hello Yamada-san, On Wed, Jan 20, 2021 at 10:06 AM Tatsuro Yamada wrote: > > Hi Julien, > > > >> Rebase only, thanks to the cfbot! V16 attached. > > > > I tested the v16 patch on a0efda88a by using "make installcheck-parallel", > > and > > my result is the following. Attached file is regression

Re: Use boolean array for nulls parameters

2021-01-19 Thread japin
On Tue, 19 Jan 2021 at 23:45, Tom Lane wrote: > japin writes: >> When I review the [1], I find that the tuple's nulls array use char type. >> However there are many places use boolean array to repsent the nulls array, >> so I think we can replace the char type nulls array to boolean type. This

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-01-19 Thread Tatsuro Yamada
Hi Julien, Rebase only, thanks to the cfbot!  V16 attached. I tested the v16 patch on a0efda88a by using "make installcheck-parallel", and my result is the following. Attached file is regression.diffs. Sorry, my environment was not suitable for the test when I sent my previous email. I fi

Re: POC: postgres_fdw insert batching

2021-01-19 Thread Amit Langote
On Wed, Jan 20, 2021 at 1:01 AM Tomas Vondra wrote: > On 1/19/21 7:23 AM, Amit Langote wrote: > > On Tue, Jan 19, 2021 at 2:06 PM tsunakawa.ta...@fujitsu.com > >> Actually, I tried to do it (adding the GetModifyBatchSize() call after > >> BeginForeignModify()), but it failed. Because > >> postg

Re: Odd, intermittent failure in contrib/pageinspect

2021-01-19 Thread Tom Lane
Andres Freund writes: > I think you don't event need checkpointer to be involved, normal buffer > replacement would do the trick. We briefly pin the page in BufferAlloc() > even if the page is clean. Longer when it's dirty, of course. True, but it seems unlikely that the pages in question here wo

Re: Odd, intermittent failure in contrib/pageinspect

2021-01-19 Thread Andres Freund
Hi, On 2021-01-18 19:40:05 -0300, Alvaro Herrera wrote: > > [ thinks for a bit... ] Does the checkpointer pin pages it's writing > > out? I guess it'd have to ... > > It does, per SyncOneBuffer(), called from BufferSync(), called from > CheckPointBuffers(). I think you don't event need checkpo

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-19 Thread Tom Lane
James Hilliard writes: > Actually, this looks path looks wrong in general, the value for > "xcrun --sdk macosx --show-sdk-path" should take precedence over > "xcrun --show-sdk-path" as the latter may be used for IOS potentially. What is "potentially"? I've found no direct means to control the SD

RE: Parallel INSERT (INTO ... SELECT ...)

2021-01-19 Thread tsunakawa.ta...@fujitsu.com
From: Tang, Haiying > Execute EXPLAIN on Patched: > Insert on public.test_part (cost=0.00..15.00 rows=0 width=0) (actual > time=44.139..44.140 rows=0 loops=1) >Buffers: shared hit=1005 read=1000 dirtied=3000 written=2000 >-> Seq Scan on public.test_data1 (cost=0.00..15.00 rows=1000 > w

RE: Add Nullif case for eval_const_expressions_mutator

2021-01-19 Thread Hou, Zhijie
Hi Thanks for the review. > It's a bit unfortunate now that between OpExpr, DistinctExpr, NullIfExpr, > and to a lesser extent ScalarArrayOpExpr we will now have several different > implementations of nearly the same thing, without any explanation why one > approach was chosen here and another th

Re: New IndexAM API controlling index vacuum strategies

2021-01-19 Thread Peter Geoghegan
On Tue, Jan 19, 2021 at 4:45 PM Peter Geoghegan wrote: > BTW, I think that increasing MaxHeapTuplesPerPage will make it > necessary to consider tidbitmap.c. Comments at the top of that file > say that it is assumed that MaxHeapTuplesPerPage is about 256. So > there is a risk of introducing perform

Re: New IndexAM API controlling index vacuum strategies

2021-01-19 Thread Peter Geoghegan
On Tue, Jan 19, 2021 at 2:57 PM Peter Geoghegan wrote: > * Maybe it would be better if you just changed the definition such > that "MAXALIGN(SizeofHeapTupleHeader)" became "MAXIMUM_ALIGNOF", with > no other changes? (Some variant of this suggestion might be better, > not sure.) > > For some reason

Re: Support for NSS as a libpq TLS backend

2021-01-19 Thread Jacob Champion
On Tue, 2021-01-19 at 21:21 +0100, Daniel Gustafsson wrote: > There is something iffy with these certs (the test fails > on mismatching ciphers and/or signature algorithms) that I haven't been able > to > pin down, but to get more eyes on this I'm posting the patch with the test > enabled. Removi

Re: Add table access method as an option to pgbench

2021-01-19 Thread David Zhang
On 2021-01-15 1:22 p.m., Andres Freund wrote: Hi, On 2020-11-25 12:41:25 +0900, Michael Paquier wrote: On Tue, Nov 24, 2020 at 03:32:38PM -0800, David Zhang wrote: But, providing another option for the end user may not be a bad idea, and it might make the tests easier at some points. My firs

Re: WIP: BRIN multi-range indexes

2021-01-19 Thread Tomas Vondra
On 1/19/21 9:44 PM, John Naylor wrote: On Tue, Jan 12, 2021 at 1:42 PM Tomas Vondra mailto:tomas.von...@enterprisedb.com>> wrote: > I suspect it'd due to minmax having to decide which "ranges" to merge, > which requires repeated sorting, etc. I certainly don't dare to claim > the current alg

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-01-19 Thread Tatsuro Yamada
Hi Julien, Rebase only, thanks to the cfbot! V16 attached. I tested the v16 patch on a0efda88a by using "make installcheck-parallel", and my result is the following. Attached file is regression.diffs. 1 of 202 tests failed. The differences

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-19 Thread James Hilliard
On Tue, Jan 19, 2021 at 3:47 PM James Hilliard wrote: > > On Tue, Jan 19, 2021 at 1:54 PM Tom Lane wrote: > > > > James Hilliard writes: > > > On Tue, Jan 19, 2021 at 10:17 AM Tom Lane wrote: > > >> Ah, got it. So "xcrun --show-sdk-path" tells us the right thing (that > > >> is, it *does* give

Re: New IndexAM API controlling index vacuum strategies

2021-01-19 Thread Peter Geoghegan
On Sun, Jan 17, 2021 at 9:18 PM Masahiko Sawada wrote: > After more thought, I think that ambulkdelete needs to be able to > refer the answer to amvacuumstrategy. That way, the index can skip > bulk-deletion when lazy vacuum doesn't vacuum heap and it also doesn’t > want to do that. Makes sense.

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-19 Thread James Hilliard
On Tue, Jan 19, 2021 at 1:54 PM Tom Lane wrote: > > James Hilliard writes: > > On Tue, Jan 19, 2021 at 10:17 AM Tom Lane wrote: > >> Ah, got it. So "xcrun --show-sdk-path" tells us the right thing (that > >> is, it *does* give us a symlink to a 10.15 SDK) but by refusing to > >> believe we've g

Re: Add primary keys to system catalogs

2021-01-19 Thread Mark Rofail
I'll post tomorrow the latest rebased patch with a summary of the issues. Let's continue this in the foreign key array thread, as to not clutter this thread. Regards, Mark Rofail. On Tue, Jan 19, 2021, 11:00 PM Joel Jacobson wrote: > On Tue, Jan 19, 2021, at 18:25, Mark Rofail wrote: > >Dear Jo

Re: Odd, intermittent failure in contrib/pageinspect

2021-01-19 Thread Tom Lane
Michael Paquier writes: > On Mon, Jan 18, 2021 at 05:47:40PM -0500, Tom Lane wrote: >> So, do we have any other tests that are invoking a manual vacuum >> and assuming it won't skip any pages? By this theory, they'd >> all be failures waiting to happen. > check_heap.sql and heap_surgery.sql have

Re: cfbot building docs - serving results

2021-01-19 Thread Thomas Munro
On Wed, Jan 20, 2021 at 10:22 AM Erik Rijkers wrote: > I am wondering if the cfbot at the moment is building the docs > (html+pdf), for the patches that it tests. I suppose that it does? If > so, what happens with the resulting (doc)files? To /dev/null? They are > not available as far as I can

Re: compression libraries and CF bot

2021-01-19 Thread Tom Lane
Thomas Munro writes: > On Wed, Jan 20, 2021 at 9:56 AM Justin Pryzby wrote: >> Also, what's the process for having new libraries installed in the CI >> environment ? > I have added lz4 to the FreeBSD and Ubuntu build tasks, so we'll see > if that helps at the next periodic build or when a new pa

Re: compression libraries and CF bot

2021-01-19 Thread Thomas Munro
On Wed, Jan 20, 2021 at 9:56 AM Justin Pryzby wrote: > Do you know if the old travis build environment had liblz4 installed ? It sounds like it. > I'm asking regarding Dilip's patch, which was getting to "check world" 2 weeks > ago but now failing to even compile, not apparently due to any chang

Re: Printing backtrace of postgres processes

2021-01-19 Thread Tom Lane
Robert Haas writes: > On Tue, Jan 19, 2021 at 12:50 PM Tom Lane wrote: >> I think it's got security hazards as well. If we restricted the >> feature to cause a trace of only one process at a time, and required >> that process to be logged in as the same database user as the >> requestor (or at l

cfbot building docs - serving results

2021-01-19 Thread Erik Rijkers
Hi Thomas, I am wondering if the cfbot at the moment is building the docs (html+pdf), for the patches that it tests. I suppose that it does? If so, what happens with the resulting (doc)files? To /dev/null? They are not available as far as I can see. Would it be feasible to make them avai

Re: create table like: ACCESS METHOD

2021-01-19 Thread Justin Pryzby
On Wed, Dec 30, 2020 at 12:33:56PM +, Simon Riggs wrote: > There are no tests for the new functionality, please could you add some? Did you look at the most recent patch? +CREATE ACCESS METHOD heapdup TYPE TABLE HANDLER heap_tableam_handler; +CREATE TABLE likeam() USING heapdup; +CREATE TABLE

Re: Add primary keys to system catalogs

2021-01-19 Thread Joel Jacobson
On Tue, Jan 19, 2021, at 18:25, Mark Rofail wrote: >Dear Joel, > >I would love to start working on this again, I tried to revive the patch again >in 2019, however, I faced the same problems as >last time (detailed in the >thread) and I didn't get the support I needed to continue with this patch.

Re: [PATCH v2 1/1] Fix detection of pwritev support for OSX.

2021-01-19 Thread Tom Lane
James Hilliard writes: > I also don't think this is a weak symbol. > From the header file it is not have __attribute__((weak_import)): > ssize_t pwritev(int, const struct iovec *, int, off_t) > __DARWIN_NOCANCEL(pwritev) __API_AVAILABLE(macos(11.0), ios(14.0), > watchos(7.0), tvos(14.0)); See th

Re: {CREATE INDEX, REINDEX} CONCURRENTLY improvements

2021-01-19 Thread Matthias van de Meent
On Mon, 18 Jan 2021, 21:25 Álvaro Herrera, wrote: > > On 2021-Jan-18, Matthias van de Meent wrote: > > > Example: > > > > 1.) RI starts > > 2.) PHASE 2: filling the index: > > 2.1.) scanning the heap (live tuple is cached) > > < tuple is deleted > > < last transaction other than RI commits, only s

Re: Printing backtrace of postgres processes

2021-01-19 Thread Robert Haas
On Tue, Jan 19, 2021 at 12:50 PM Tom Lane wrote: > The thing that is scaring me the most is the "broadcast" aspect. > For starters, I think that that is going to be useless in the > field because of the likelihood that different backends' stack > traces will get interleaved in whatever log file th

compression libraries and CF bot

2021-01-19 Thread Justin Pryzby
Do you know if the old travis build environment had liblz4 installed ? I'm asking regarding Dilip's patch, which was getting to "check world" 2 weeks ago but now failing to even compile, not apparently due to any change in the patch. Also, are the historic logs available somewhere ? http://cfbot.

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-19 Thread Tom Lane
James Hilliard writes: > On Tue, Jan 19, 2021 at 10:17 AM Tom Lane wrote: >> Ah, got it. So "xcrun --show-sdk-path" tells us the right thing (that >> is, it *does* give us a symlink to a 10.15 SDK) but by refusing to >> believe we've got the right thing, we end up picking MacOSX11.1.sdk. >> Drat

Re: Some coverage for DROP OWNED BY with pg_default_acl

2021-01-19 Thread Alvaro Herrera
On 2021-Jan-19, Michael Paquier wrote: > And while reviewing the thing, I have spotted that there is a specific > path for pg_default_acl in RemoveRoleFromObjectACL() that has zero > coverage. This can be triggered with DROP OWNED BY, and it is > actually safe to run as long as this is done in a

Re: [Patch] ALTER SYSTEM READ ONLY

2021-01-19 Thread Robert Haas
On Thu, Jan 14, 2021 at 6:29 AM Amul Sul wrote: > To move development, testing, and the review forward, I have commented out the > code acquiring CheckpointLock from CreateCheckPoint() in the 0003 patch and > added the changes for the checkpointer so that system read-write state change > request c

Re: WIP: BRIN multi-range indexes

2021-01-19 Thread John Naylor
On Tue, Jan 12, 2021 at 1:42 PM Tomas Vondra wrote: > I suspect it'd due to minmax having to decide which "ranges" to merge, > which requires repeated sorting, etc. I certainly don't dare to claim > the current algorithm is perfect. I wouldn't have expected such big > difference, though - so defin

Re: Support for NSS as a libpq TLS backend

2021-01-19 Thread Daniel Gustafsson
> On 4 Dec 2020, at 01:57, Jacob Champion wrote: > > On Nov 17, 2020, at 7:00 AM, Daniel Gustafsson wrote: >> >> Nice, thanks for the fix! I've incorporated your patch into the attached v20 >> which also fixes client side error reporting to be more readable. > > I was testing handshake failur

Re: Add primary keys to system catalogs

2021-01-19 Thread Tom Lane
Laurenz Albe writes: > On Sun, 2021-01-17 at 17:07 -0500, Tom Lane wrote: >> [...] do we really want to prefer >> using the OID indexes as the primary keys? In most cases there's some >> other index that seems to me to be what a user would think of as the >> pkey, for example pg_class_relname_nsp

Re: Change default of checkpoint_completion_target

2021-01-19 Thread Tom Lane
Stephen Frost writes: > In passing, I noticed that we have a lot of documentation like: > This parameter can only be set in the postgresql.conf file or on the > server command line. > ... which hasn't been true since the introduction of ALTER SYSTEM. Well, it's still true if you understand "the

Re: Change default of checkpoint_completion_target

2021-01-19 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > Any further comments or thoughts on this one? > > This: > > +total time between checkpoints. The default is 0.9, which spreads the > +checkpoint across the entire checkpoint timeout period of time, >

Re: Change default of checkpoint_completion_target

2021-01-19 Thread Tom Lane
Stephen Frost writes: > Any further comments or thoughts on this one? This: +total time between checkpoints. The default is 0.9, which spreads the +checkpoint across the entire checkpoint timeout period of time, is confusing because 0.9 is obviously not 1.0; people will wonder w

Re: Change default of checkpoint_completion_target

2021-01-19 Thread Stephen Frost
Greetings, * Peter Eisentraut (peter.eisentr...@enterprisedb.com) wrote: > On 2021-01-13 23:10, Stephen Frost wrote: > >>Yes, I agree, and am involved in that thread as well- currently waiting > >>feedback from others about the proposed approach. > >I've tried to push that forward. I'm happy to u

Re: [HACKERS] Custom compression methods

2021-01-19 Thread Justin Pryzby
Thanks for updating the patch. On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby wrote: > The most recent patch doesn't compile --without-lz4: On Tue, Jan 05, 2021 at 11:19:33AM +0530, Dilip Kumar wrote: > On Mon, Jan 4, 2021 at 10:08 PM Justin Pryzby wrote: > > I think I first saw it on cfbot and I

Re: Add docs stub for recovery.conf

2021-01-19 Thread Stephen Frost
Greetings, * Stephen Frost (sfr...@snowman.net) wrote: > * Craig Ringer (craig.rin...@enterprisedb.com) wrote: > > On Thu, 14 Jan 2021 at 03:44, Stephen Frost wrote: > > > Alright, how does this look? The new entries are all under the > > > 'obsolete' section to keep it out of the main line, but

Re: Add docs stub for recovery.conf

2021-01-19 Thread Stephen Frost
Greetings, * Craig Ringer (craig.rin...@enterprisedb.com) wrote: > On Thu, 14 Jan 2021 at 03:44, Stephen Frost wrote: > > Alright, how does this look? The new entries are all under the > > 'obsolete' section to keep it out of the main line, but should work to > > 'fix' the links that currently 4

Re: [HACKERS] [PATCH] Generic type subscripting

2021-01-19 Thread Pavel Stehule
Hi I found minor issues. Doc - missing tag and three whitespaces issues see attached patch Following sentence is hard to read due long nested example If the + path contradicts structure of modified jsonb for any individual + value (e.g. path val['a']['b']['c'] assumes keys + 'a' and 'b'

Re: Rethinking plpgsql's assignment implementation

2021-01-19 Thread Pavel Stehule
Hi Now, I am testing subscribing on the jsonb feature, and I found one issue, that is not supported by parser. When the target is scalar, then all is ok. But we can have a plpgsql array of jsonb values. postgres=# do $$ declare j jsonb[]; begin j[1] = '{"b":"Ahoj"}'; raise notice '%', j; r

Re: Add primary keys to system catalogs

2021-01-19 Thread Mark Rofail
Dear Joel, I would love to start working on this again, I tried to revive the patch again in 2019, however, I faced the same problems as last time (detailed in the thread) and I didn't get the support I needed to continue with this patch. Are you willing to help rebase and finally publish this pa

Re: [PATCH v2 1/1] Fix detection of pwritev support for OSX.

2021-01-19 Thread James Hilliard
On Tue, Jan 19, 2021 at 10:29 AM Tom Lane wrote: > > James Hilliard writes: > > Fixes: > > fd.c:3661:10: warning: 'pwritev' is only available on macOS 11.0 or newer > > [-Wunguarded-availability-new] > > It's still missing preadv, and it still has nonzero chance of breaking > successful detectio

Re: simplifying foreign key/RI checks

2021-01-19 Thread Corey Huinker
> > I decided not to deviate from pk_ terminology so that the new code > doesn't look too different from the other code in the file. Although, > I guess we can at least call the main function > ri_ReferencedKeyExists() instead of ri_PrimaryKeyExists(), so I've > changed that. > I agree with leavi

Re: Printing backtrace of postgres processes

2021-01-19 Thread Tom Lane
Robert Haas writes: > On Sat, Jan 16, 2021 at 3:21 PM Tom Lane wrote: >> (Personally, I think this whole patch fails the safety-vs-usefulness >> tradeoff, but I expect I'll get shouted down.) > What do you see as the main safety risks here? The thing that is scaring me the most is the "broadcas

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-19 Thread James Hilliard
On Tue, Jan 19, 2021 at 10:17 AM Tom Lane wrote: > > James Hilliard writes: > > On Tue, Jan 19, 2021 at 8:57 AM Tom Lane wrote: > >> It worked for me and for Sergey, so we need to figure out what's different > >> about your setup. What do you get from "xcrun --show-sdk-path" and > >> "xcrun --s

Re: New Table Access Methods for Multi and Single Inserts

2021-01-19 Thread Jeff Davis
On Mon, 2021-01-18 at 08:58 +0100, Luc Vlaming wrote: > You mean how it could because of that the table modification API > uses > the table_tuple_insert_speculative ? Just wondering if you think if > it > generally cannot work or would like to see that path / more paths > integrated in to the pa

Re: Printing backtrace of postgres processes

2021-01-19 Thread Robert Haas
On Sat, Jan 16, 2021 at 3:21 PM Tom Lane wrote: > I'd argue that backtraces for those processes aren't really essential, > and indeed that trying to make the syslogger report its own backtrace > is damn dangerous. I agree. Ideally I'd like to be able to use the same mechanism everywhere and inclu

Re: [PATCH v2 1/1] Fix detection of pwritev support for OSX.

2021-01-19 Thread Tom Lane
James Hilliard writes: > Fixes: > fd.c:3661:10: warning: 'pwritev' is only available on macOS 11.0 or newer > [-Wunguarded-availability-new] It's still missing preadv, and it still has nonzero chance of breaking successful detection of pwritev on platforms other than yours, and it's still really

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-19 Thread Tom Lane
James Hilliard writes: > On Tue, Jan 19, 2021 at 8:57 AM Tom Lane wrote: >> It worked for me and for Sergey, so we need to figure out what's different >> about your setup. What do you get from "xcrun --show-sdk-path" and >> "xcrun --sdk macosx --show-sdk-path"? What have you got under >> /Libra

Re: psql \df choose functions by their arguments

2021-01-19 Thread Greg Sabino Mullane
Ha ha ha, my bad, I am not sure why I left those out. Here is a new patch with int2, int4, and int8. Thanks for the email. Cheers, Greg v6-psql-df-pick-function-by-type.patch Description: Binary data

[PATCH v2 1/1] Fix detection of pwritev support for OSX.

2021-01-19 Thread James Hilliard
Fixes: gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -O2 -I../../../../src/include -isysroot /Applications/Xcode.app/Contents/De

Re: search_plan_tree(): handling of non-leaf CustomScanState nodes causes segfault

2021-01-19 Thread Tom Lane
I wrote: > Now that I look at this, I strongly wonder whether whoever added > MergeAppend support here understood what they were doing. That > looks broken, because child nodes will typically be positioned on > tuples, whether or not the current top-level output came from them. > So I fear we coul

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-19 Thread James Hilliard
On Tue, Jan 19, 2021 at 8:57 AM Tom Lane wrote: > > James Hilliard writes: > > On Tue, Jan 19, 2021 at 8:27 AM Tom Lane wrote: > >> We already dealt with that by not selecting an SDK newer than the > >> underlying OS (see 4823621db). > > > Tried that, doesn't work, not even sure how it could pos

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-01-19 Thread Julien Rouhaud
On Fri, Jan 8, 2021 at 1:07 AM Julien Rouhaud wrote: > > v15 that fixes recent conflicts. Rebase only, thanks to the cfbot! V16 attached. From a0388c53d9755cfd706513f7f02a08b31a48aacb Mon Sep 17 00:00:00 2001 From: Julien Rouhaud Date: Mon, 18 Mar 2019 18:55:50 +0100 Subject: [PATCH v16 2/3] Ex

Re: pg_class.reltype -> pg_type.oid missing for pg_toast table

2021-01-19 Thread Tom Lane
"Joel Jacobson" writes: > When copying all tables in pg_catalog, to a separate schema with the purpose > of testing if foreign keys could be added for all oid columns, I got an error > for a toast table: > ERROR: insert or update on table "pg_class" violates foreign key constraint > "pg_class_r

Re: popcount

2021-01-19 Thread Isaac Morland
On Tue, 19 Jan 2021 at 11:38, David Fetter wrote: > You bring up an excellent point, which is that our builtin functions > could use a lot more documentation directly to hand than they now > have. For example, there's a lot of needless ambiguity created by > function comments which leave it up i

Re: popcount

2021-01-19 Thread David Fetter
On Tue, Jan 19, 2021 at 07:58:12AM -0500, Robert Haas wrote: > On Tue, Jan 19, 2021 at 3:06 AM Peter Eisentraut > wrote: > > On 2021-01-18 16:34, Tom Lane wrote: > > > Peter Eisentraut writes: > > >> [ assorted nits ] > > > > > > At the level of bikeshedding ... I quite dislike using the name "po

pg_class.reltype -> pg_type.oid missing for pg_toast table

2021-01-19 Thread Joel Jacobson
I have a memory of the catalog not being MVCC, so maybe this is normal and expected, but I wanted to report it in case it's not. When copying all tables in pg_catalog, to a separate schema with the purpose of testing if foreign keys could be added for all oid columns, I got an error for a toast t

Re: Stronger safeguard for archive recovery not to miss data

2021-01-19 Thread Laurenz Albe
On Mon, 2021-01-18 at 07:34 +, osumi.takami...@fujitsu.com wrote: > I noticed that this message should cover both archive recovery modes, > which means in recovery mode and standby mode. Then, I combined your > suggestion above with this point of view. Have a look at the updated patch. > I also

Re: Is Recovery actually paused?

2021-01-19 Thread Dilip Kumar
On Tue, Jan 19, 2021 at 8:34 AM Dilip Kumar wrote: > > On Tue, 19 Jan 2021 at 8:12 AM, Yugo NAGATA wrote: >> >> On Sun, 17 Jan 2021 11:33:52 +0530 >> Dilip Kumar wrote: >> >> > On Thu, Jan 14, 2021 at 6:49 PM Yugo NAGATA wrote: >> > > >> > > On Wed, 13 Jan 2021 17:49:43 +0530 >> > > Dilip Kumar

Re: POC: postgres_fdw insert batching

2021-01-19 Thread Tomas Vondra
On 1/19/21 7:23 AM, Amit Langote wrote: On Tue, Jan 19, 2021 at 2:06 PM tsunakawa.ta...@fujitsu.com wrote: From: Amit Langote I apologize in advance for being maybe overly pedantic, but I noticed that, in ExecInitModifyTable(), you decided to place the call outside the loop that goes over

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-19 Thread Tom Lane
James Hilliard writes: > On Tue, Jan 19, 2021 at 8:27 AM Tom Lane wrote: >> We already dealt with that by not selecting an SDK newer than the >> underlying OS (see 4823621db). > Tried that, doesn't work, not even sure how it could possibly fix this > issue at all, It worked for me and for Serge

Re: Use boolean array for nulls parameters

2021-01-19 Thread Tom Lane
japin writes: > When I review the [1], I find that the tuple's nulls array use char type. > However there are many places use boolean array to repsent the nulls array, > so I think we can replace the char type nulls array to boolean type. This > change will break the SPI_xxx API, I'm not sure whe

Re: TOAST condition for column size

2021-01-19 Thread Tom Lane
Dilip Kumar writes: > On Tue, 19 Jan 2021 at 6:28 PM, Amit Kapila wrote: >> Won't it be safe because we don't align individual attrs of type >> varchar where length is less than equal to 127? > Yeah right, I just missed that point. Yeah, the minimum on biggest_size has nothing to do with align

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-19 Thread James Hilliard
On Tue, Jan 19, 2021 at 8:27 AM Tom Lane wrote: > > James Hilliard writes: > > Fixes: > > gcc -Wall -Wmissing-prototypes -Wpointer-arith > > -Wdeclaration-after-statement -Werror=vla -Wendif-labels > > -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv > > -Wno-unused-co

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-19 Thread Tom Lane
James Hilliard writes: > Fixes: > gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement > -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wformat-security > -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -O2 > -I../../../../src/include -isysroot >

Re: search_plan_tree(): handling of non-leaf CustomScanState nodes causes segfault

2021-01-19 Thread Tom Lane
David Geier writes: > On 18.01.21 23:42, Tom Lane wrote: >> OK, cool. I was afraid you'd argue that you really needed your CustomScan >> node to be transparent in such cases. We could imagine inventing an >> additional custom-scan-provider callback to embed the necessary knowledge, >> but I'd ra

Re: Use boolean array for nulls parameters

2021-01-19 Thread Hamid Akhtar
I personally don't see any benefit in this change. The focus shouldn't be on fixing things that aren't broken. Perhaps, there is more value in using bitmap data type to keep track of NULL values, which is typical storage vs performance debate, and IMHO, it's better to err on using slightly more sto

Re: ResourceOwner refactoring

2021-01-19 Thread Heikki Linnakangas
On 18/01/2021 16:34, Alvaro Herrera wrote: On 2021-Jan-18, Heikki Linnakangas wrote: +static ResourceOwnerFuncs jit_funcs = +{ + /* relcache references */ + .name = "LLVM JIT context", + .phase = RESOURCE_RELEASE_BEFORE_LOCKS, + .ReleaseResource = ResOwnerReleaseJitConte

Re: [PATCH] More docs on what to do and not do in extension code

2021-01-19 Thread Bharath Rupireddy
On Mon, Jan 18, 2021 at 1:27 PM Craig Ringer wrote: > Hi folks > > The attached patch expands the xfunc docs and bgworker docs a little, > providing a starting point for developers to learn how to do some common > tasks the Postgres Way. > > It mentions in brief these topics: > > * longjmp() bas

Re: [HACKERS] [PATCH] Generic type subscripting

2021-01-19 Thread Dmitry Dolgov
> On Thu, Jan 14, 2021 at 12:02:42PM -0500, Dian M Fay wrote: > > Other than that, since I've already posted the patch for returning an > > error option, it seems that the only thing left is to decide with which > > version to go. > > The trigger issue (which I did verify) makes the "no update" opt

  1   2   >