Re: [PATCH v1] elog.c: Remove special case which avoided %*s format strings..

2020-08-04 Thread Michael Paquier
On Sun, Aug 02, 2020 at 11:59:48PM -0500, Justin Pryzby wrote: > ..which should no longer be needed since it was a performance hack for > specific > platform snprintf, which are no longer used. Did you check if our implementation of src/port/snprintf.c makes %*s much slower than %s or not? FWIW,

Re: EDB builds Postgres 13 with an obsolete ICU version

2020-08-04 Thread Dave Page
On Tue, Aug 4, 2020 at 1:04 AM Bruce Momjian wrote: > On Mon, Aug 3, 2020 at 08:56:06PM +0200, Daniel Verite wrote: > > Hi, > > > > As a follow-up to bug #16570 [1] and other previous discussions > > on the mailing-lists, I'm checking out PG13 beta for Windows > > from: > > https://www.enterpr

LSM tree for Postgres

2020-08-04 Thread Konstantin Knizhnik
Hi hackers, I want to share results of my last research of implementing LSM index in Postgres. Most of modern databases (RocksDB, MongoDB, Tarantool,...) are using LSM tree instead of classical B-Tree. From one side, capacity of RAM at modern servers allows to keep the whole database in memo

Rewrite view?

2020-08-04 Thread Hao Wu
Hi hackers, When I define a view relevant with `char varying`, it shows the type is casted to text. See, gpadmin=# CREATE TABLE foobar (a character varying); CREATE TABLE gpadmin=# CREATE VIEW fooview AS SELECT * FROM foobar WHERE a::character varying = ANY(ARRAY['foo'::character varying, 'bar'

Re: Is it worth accepting multiple CRLs?

2020-08-04 Thread Kyotaro Horiguchi
At Mon, 03 Aug 2020 16:20:40 +0900 (JST), Kyotaro Horiguchi wrote in > Thanks for the opinion. I'll continue working on this. This is it, but.. Looking closer I realized that certificates are verified in each backend so CRL cache doesn't work at all for the hashed directory method. Therefore,

Re: SSL TAP test fails due to default client certs.

2020-08-04 Thread Kyotaro Horiguchi
At Tue, 4 Aug 2020 14:43:58 +0900, Michael Paquier wrote in > On Tue, Aug 04, 2020 at 12:00:33PM +0900, Kyotaro Horiguchi wrote: > > While poking at ssl code, I noticed that 002_scram.pl fails if > > ~/.postgresql/root.crt exists. This has been fixed once but > > d6e612f837 reintroduced one. Th

Re: [PATCH v1] elog.c: Remove special case which avoided %*s format strings..

2020-08-04 Thread David Rowley
On Tue, 4 Aug 2020 at 19:36, Michael Paquier wrote: > Did you check if our implementation of src/port/snprintf.c makes %*s > much slower than %s or not? FWIW, I have just run a small test on my > laptop, and running 100M calls of snprintf() with "%s" in a tight loop > takes 2.7s, with "%*s" and a

Re: [PATCH] - Provide robust alternatives for replace_string

2020-08-04 Thread Asim Praveen
> On 03-Aug-2020, at 8:36 PM, Alvaro Herrera wrote: > > On 2020-Aug-03, Asim Praveen wrote: > >> Thank you Alvaro for reviewing the patch! >> >>> On 01-Aug-2020, at 7:22 AM, Alvaro Herrera wrote: >>> >>> What happens if a replacement string happens to be split in the middle >>> by the fgets

Re: EDB builds Postgres 13 with an obsolete ICU version

2020-08-04 Thread Thomas Kellerer
Dave Page schrieb am 04.08.2020 um 10:06: > Correct - updating ICU would mean a reindex is required following any > upgrade, major or minor. > > I would really like to find an acceptable solution to this however as > it really would be good to be able to update ICU. > What about providing a newer

Re: EDB builds Postgres 13 with an obsolete ICU version

2020-08-04 Thread Magnus Hagander
On Tue, Aug 4, 2020 at 10:07 AM Dave Page wrote: > > > On Tue, Aug 4, 2020 at 1:04 AM Bruce Momjian wrote: > >> On Mon, Aug 3, 2020 at 08:56:06PM +0200, Daniel Verite wrote: >> > Hi, >> > >> > As a follow-up to bug #16570 [1] and other previous discussions >> > on the mailing-lists, I'm checki

Re: EDB builds Postgres 13 with an obsolete ICU version

2020-08-04 Thread Dave Page
On Tue, Aug 4, 2020 at 10:29 AM Magnus Hagander wrote: > On Tue, Aug 4, 2020 at 10:07 AM Dave Page wrote: > >> >> >> On Tue, Aug 4, 2020 at 1:04 AM Bruce Momjian wrote: >> >>> On Mon, Aug 3, 2020 at 08:56:06PM +0200, Daniel Verite wrote: >>> > Hi, >>> > >>> > As a follow-up to bug #16570 [1]

Re: FailedAssertion("pd_idx == pinfo->nparts", File: "execPartition.c", Line: 1689)

2020-08-04 Thread Amit Langote
On Tue, Aug 4, 2020 at 1:11 AM Justin Pryzby wrote: > > On Mon, Aug 03, 2020 at 11:41:37AM -0400, Robert Haas wrote: > > On Sun, Aug 2, 2020 at 2:11 PM Justin Pryzby wrote: > > > Based on commit logs, I suspect this may be an "older bug", specifically > > > maybe > > > with: > > > > > > |commit

Re: Can a background worker exist without shared memory access for EXEC_BACKEND cases?

2020-08-04 Thread Bharath Rupireddy
Thank you Robert for the comments. On Mon, Aug 3, 2020 at 9:19 PM Robert Haas wrote: > > On Fri, Jul 31, 2020 at 11:13 PM Bharath Rupireddy > wrote: > > memory. MyLatch variable also gets created in shared mode. And having > > no shared memory access for the worker for EXEC_BACKEND cases(in > >

Re: problem with RETURNING and update row movement

2020-08-04 Thread Etsuro Fujita
Amit-san, On Mon, Aug 3, 2020 at 2:55 PM Amit Langote wrote: > However, I noticed that having system columns like ctid, xmin, etc. in > the RETURNING list is now broken and maybe irrepairably due to the > approach we are taking in the patch. Let me show an example: > > drop table foo; > create t

Yet another issue with step generation in partition pruning

2020-08-04 Thread Etsuro Fujita
Commit 13838740f fixed some issues with step generation in partition pruning, but as I mentioned in [1], I noticed that there is yet another issue: get_steps_using_prefix() assumes that clauses in the passed-in prefix list are sorted in ascending order of their partition key numbers, but the caller

Re: Reduce/eliminate the impact of FPW

2020-08-04 Thread Stephen Frost
Greetings, Please don't top-post on these lists. * SATYANARAYANA NARLAPURAM (satyanarlapu...@gmail.com) wrote: > Increasing checkpoint_timeout helps reduce the amount of log written to the > disk. This has several benefits like, reduced number of WAL IO, archival > load on the system, less networ

Re: Replace remaining StrNCpy() by strlcpy()

2020-08-04 Thread Peter Eisentraut
On 2020-08-03 19:39, Tom Lane wrote: That's easy to fix, but it's perhaps wondering briefly why it needs to be zero-padded. hashname() doesn't care, heap_form_tuple() doesn't care. Does anything care? We do have an expectation that there are no undefined bytes in values to be stored on-disk.

Re: Since '2001-09-09 01:46:40'::timestamp microseconds are lost when extracting epoch

2020-08-04 Thread Peter Eisentraut
On 2020-05-25 15:28, Peter Eisentraut wrote: On 2019-12-02 23:52, Thomas Munro wrote: I'm not an expert in floating point math but hopefully it means that no type change is required - double precision can handle it. Me neither, but the SQL standard requires us to use an exact numeric type, so i

Re: Since '2001-09-09 01:46:40'::timestamp microseconds are lost when extracting epoch

2020-08-04 Thread Pavel Stehule
út 4. 8. 2020 v 16:08 odesílatel Peter Eisentraut < peter.eisentr...@2ndquadrant.com> napsal: > On 2020-05-25 15:28, Peter Eisentraut wrote: > > On 2019-12-02 23:52, Thomas Munro wrote: > >>> I'm not an expert in floating point math but hopefully it means that no > >>> type change is required - do

Re: Concurrency bug in amcheck

2020-08-04 Thread Alexander Korotkov
Hi, Peter! On Sat, Aug 1, 2020 at 3:23 AM Peter Geoghegan wrote: > On Wed, May 13, 2020 at 4:06 PM Peter Geoghegan wrote: > > On Mon, May 11, 2020 at 5:56 AM Alexander Korotkov > > wrote: > > > Thank you. 2nd patch is proposed for master and makes btree page > > > unlink remove all the items

Re: WIP: BRIN multi-range indexes

2020-08-04 Thread Alexander Korotkov
Hi, Tomas! Sorry for the late reply. On Sun, Jul 19, 2020 at 6:19 PM Tomas Vondra wrote: > I think there's a number of weak points in this approach. > > Firstly, it assumes the summaries can be represented as arrays of > built-in types, which I'm not really sure about. It clearly is not true > f

Re: new heapcheck contrib module

2020-08-04 Thread Robert Haas
On Mon, Aug 3, 2020 at 1:16 PM Peter Geoghegan wrote: > If you really wanted to do this, > you'd have to describe a practical scenario under which it made sense > to soldier on, where we'd definitely be able to count the number of > problems in a meaningful way, without much risk of either massive

Re: LSM tree for Postgres

2020-08-04 Thread Alexander Korotkov
Hi! On Tue, Aug 4, 2020 at 11:22 AM Konstantin Knizhnik wrote: > I want to share results of my last research of implementing LSM index in > Postgres. > Most of modern databases (RocksDB, MongoDB, Tarantool,...) are using LSM tree > instead of classical B-Tree. I wouldn't say it that way. I wo

Re: FailedAssertion("pd_idx == pinfo->nparts", File: "execPartition.c", Line: 1689)

2020-08-04 Thread Justin Pryzby
On Tue, Aug 04, 2020 at 08:12:10PM +0900, Amit Langote wrote: > It may be this commit that went into PG 12 that is causing the problem: Thanks for digging into this. > to account for partitions that were pruned by the planner for which we > decided to put 0 into relid_map, but it only considered

Re: LSM tree for Postgres

2020-08-04 Thread Tomas Vondra
On Tue, Aug 04, 2020 at 11:22:13AM +0300, Konstantin Knizhnik wrote: Hi hackers, I want to share results of my last research of implementing LSM index in Postgres. Most of modern databases (RocksDB, MongoDB, Tarantool,...) are using LSM tree instead of classical B-Tree. I was under the imp

Re: WIP: BRIN multi-range indexes

2020-08-04 Thread Tomas Vondra
On Tue, Aug 04, 2020 at 05:36:51PM +0300, Alexander Korotkov wrote: Hi, Tomas! Sorry for the late reply. On Sun, Jul 19, 2020 at 6:19 PM Tomas Vondra wrote: I think there's a number of weak points in this approach. Firstly, it assumes the summaries can be represented as arrays of built-in ty

Re: LSM tree for Postgres

2020-08-04 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > On Tue, Aug 04, 2020 at 11:22:13AM +0300, Konstantin Knizhnik wrote: > >two top indexes used in cyclic way and one main index. When top index > >reaches some threshold value > >we initiate merge with main index, done by bgworker and

Re: LSM tree for Postgres

2020-08-04 Thread Alexander Korotkov
On Tue, Aug 4, 2020 at 6:11 PM Tomas Vondra wrote: > On Tue, Aug 04, 2020 at 11:22:13AM +0300, Konstantin Knizhnik wrote: > >Hi hackers, > > > >I want to share results of my last research of implementing LSM index > >in Postgres. > >Most of modern databases (RocksDB, MongoDB, Tarantool,...) are us

Re: new heapcheck contrib module

2020-08-04 Thread Peter Geoghegan
On Tue, Aug 4, 2020 at 7:59 AM Robert Haas wrote: > I think we should try not to imagine anything in particular. Just to > be clear, I am not trying to knock what you have; I know it was a lot > of work to create and it's a huge improvement over having nothing. But > in my mind, a perfect tool wou

Re: new heapcheck contrib module

2020-08-04 Thread Robert Haas
On Fri, Jul 31, 2020 at 12:33 PM Andres Freund wrote: > I'm not sure what I was thinking "back then", but right now I'd argue > that the best lock against vacuum isn't a SUE, but announcing the > correct ->xmin, so you can be sure that clog entries won't be yanked out > from under you. Potentially

Re: Error on failed COMMIT

2020-08-04 Thread Dave Cramer
Attached is the rebased patch for consideration. Dave Cramer www.postgres.rocks > > 0001-Throw-error-and-rollback-on-a-failed-transaction-ins.patch Description: Binary data

Re: Parallel copy

2020-08-04 Thread Tomas Vondra
On Mon, Aug 03, 2020 at 12:33:48PM +0530, Bharath Rupireddy wrote: On Sat, Aug 1, 2020 at 9:55 AM vignesh C wrote: The patches were not applying because of the recent commits. I have rebased the patch over head & attached. I rebased v2-0006-Parallel-Copy-For-Binary-Format-Files.patch. Putti

Re: Amcheck: do rightlink verification with lock coupling

2020-08-04 Thread Andrey M. Borodin
> 4 авг. 2020 г., в 03:44, Peter Geoghegan написал(а): > > On Mon, Jul 20, 2020 at 11:46 AM Andrey M. Borodin > wrote: >> In this thread [0] we decided that lock coupling is necessary for >> btree_xlog_unlink_page(). >> So, maybe let's revive this patch? > > Yes, let's do that. Can you resu

Re: new heapcheck contrib module

2020-08-04 Thread Robert Haas
On Tue, Aug 4, 2020 at 12:00 PM Peter Geoghegan wrote: > With indexes you tend to have redundancy in how relationships among > pages are described. So you have siblings whose pointers must be in > agreement (left points to right, right points to left), and it's not > clear which one you should tru

Re: Can a background worker exist without shared memory access for EXEC_BACKEND cases?

2020-08-04 Thread Robert Haas
On Tue, Aug 4, 2020 at 7:27 AM Bharath Rupireddy wrote: > I could get these points earlier in my initial analysis. In fact, I > could figure out the flow on Windows, how these parameters are shared > using a shared file(CreateFileMapping(), MapViewOfFile()), and the > shared file name being passed

Re: Confusing behavior of create table like

2020-08-04 Thread Robert Haas
On Mon, Aug 3, 2020 at 12:35 PM Peter Eisentraut wrote: > On 2020-08-03 14:58, Konstantin Knizhnik wrote: > > May be generated and identity columns are good things. I have nothing > > against them. > > But what preventing us from providing the similar behavior for > > serial/bigseries types? > > I

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2020-08-04 Thread Alvaro Herrera
On 2020-Aug-03, Alvaro Herrera wrote: > Why two transactions? The reason is that in order for this to work, we > make a catalog change (mark it detached), and commit so that all > concurrent transactions can see the change. A second transaction waits > for anybody who holds any lock on the parti

Re: LSM tree for Postgres

2020-08-04 Thread Konstantin Knizhnik
On 04.08.2020 18:04, Alexander Korotkov wrote: Hi! On Tue, Aug 4, 2020 at 11:22 AM Konstantin Knizhnik wrote: I want to share results of my last research of implementing LSM index in Postgres. Most of modern databases (RocksDB, MongoDB, Tarantool,...) are using LSM tree instead of classic

Re: LSM tree for Postgres

2020-08-04 Thread Konstantin Knizhnik
On 04.08.2020 18:11, Tomas Vondra wrote: On Tue, Aug 04, 2020 at 11:22:13AM +0300, Konstantin Knizhnik wrote: Hi hackers, I want to share results of my last research of implementing LSM index in Postgres. Most of modern databases (RocksDB, MongoDB, Tarantool,...) are using LSM tree instead

Re: LSM tree for Postgres

2020-08-04 Thread Konstantin Knizhnik
On 04.08.2020 18:18, Stephen Frost wrote: Independently while considering this, I don't think the issues around how to deal with unique btrees properly has really been considered- you certainly can't stop your search on the first tuple you find even if the index is unique, since the "unique"

Re: Yet another fast GiST build (typo)

2020-08-04 Thread Andrey M. Borodin
> 30 июля 2020 г., в 06:26, Thomas Munro написал(а): > > On Fri, Jul 10, 2020 at 6:55 PM Andrey M. Borodin > wrote: >> Thanks! Fixed. > > It's not a bug, but I think those 64 bit constants should be wrapped > in UINT64CONST(), following our convention. Thanks, fixed! > I'm confused about th

Re: Confusing behavior of create table like

2020-08-04 Thread Konstantin Knizhnik
On 04.08.2020 19:53, Robert Haas wrote: On Mon, Aug 3, 2020 at 12:35 PM Peter Eisentraut wrote: On 2020-08-03 14:58, Konstantin Knizhnik wrote: May be generated and identity columns are good things. I have nothing against them. But what preventing us from providing the similar behavior for

Re: LSM tree for Postgres

2020-08-04 Thread Tomas Vondra
On Tue, Aug 04, 2020 at 08:18:01PM +0300, Konstantin Knizhnik wrote: On 04.08.2020 18:11, Tomas Vondra wrote: On Tue, Aug 04, 2020 at 11:22:13AM +0300, Konstantin Knizhnik wrote: Hi hackers, I want to share results of my last research of implementing LSM index in Postgres. Most of modern da

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2020-08-04 Thread Robert Haas
On Mon, Aug 3, 2020 at 7:49 PM Alvaro Herrera wrote: > Why two transactions? The reason is that in order for this to work, we > make a catalog change (mark it detached), and commit so that all > concurrent transactions can see the change. A second transaction waits > for anybody who holds any lo

Re: LSM tree for Postgres

2020-08-04 Thread Konstantin Knizhnik
On 04.08.2020 20:44, Tomas Vondra wrote: Unique indexes are not supported now. And I do not see some acceptable solution here. If we will have to check presence of duplicate at the time of insert then it will eliminate all advantages of LSM approach. And if we postpone to the moment of merge

For standby pg_ctl doesn't wait for PM_STATUS_READY in presence of promote_trigger_file

2020-08-04 Thread Ashwin Agrawal
If shutdown (non hot enabled) standby and promote the standby using promote_trigger_file via pg_ctl start with -w (wait), currently pg_ctl returns as soon as recovery is started. Instead would be helpful if pg_ctl can wait till PM_STATUS_READY for this case, given promotion is requested. pg_ctl -w

Re: display offset along with block number in vacuum errors

2020-08-04 Thread Mahendra Singh Thalor
Thanks Sawada and Justin. On Sun, 2 Aug 2020 at 09:33, Masahiko Sawada wrote: > > On Sat, 1 Aug 2020 at 16:02, Mahendra Singh Thalor wrote: > > > > Thanks Justin. > > > > On Sat, 1 Aug 2020 at 11:47, Justin Pryzby wrote: > > > > > > On Fri, Jul 31, 2020 at 04:55:14PM -0500, Justin Pryzby wrote:

Re: FailedAssertion("pd_idx == pinfo->nparts", File: "execPartition.c", Line: 1689)

2020-08-04 Thread Robert Haas
On Mon, Aug 3, 2020 at 12:11 PM Justin Pryzby wrote: > (gdb) p *pinfo->relid_map@414 > (gdb) p *partdesc->oids@415 Whoa, I didn't know about using @ in gdb to print multiple elements. Wild! Anyway, these two arrays differ in that the latter array has 22653702 inserted at the beginning and 226288

Re: 13dev failed assert: comparetup_index_btree(): ItemPointer values should never be equal

2020-08-04 Thread Tom Lane
Peter Geoghegan writes: > Any further thoughts on this, Tom? Sorry for slow response ... my ISP had an equipment failure that took out my email service for most of a day. > This is clearly a live bug that we should fix before too long. I could > write the patch myself, but I would like to get yo

Re: Support for NSS as a libpq TLS backend

2020-08-04 Thread Daniel Gustafsson
> On 3 Aug 2020, at 21:18, Andrew Dunstan > wrote: > On 8/3/20 12:46 PM, Andrew Dunstan wrote: >> On 7/31/20 4:44 PM, Andrew Dunstan wrote: >>> OK, here is an update of your patch that compiles and runs against NSS >>> under Windows (VS2019). Out of curiosity since I'm not familiar with Windows

Re: Confusing behavior of create table like

2020-08-04 Thread Peter Eisentraut
On 2020-08-04 19:36, Konstantin Knizhnik wrote: Yes, generated columns are also using implicitly generated sequences. So them are  very similar with SERIAL/BIGSERIAL columns. This actually make we wonder why we can not handle them in the same way in CREATE TABLE LIKE. The current specification

Re: 13dev failed assert: comparetup_index_btree(): ItemPointer values should never be equal

2020-08-04 Thread Peter Geoghegan
On Tue, Aug 4, 2020 at 1:31 PM Tom Lane wrote: > The core of the issue seems to be that in the presence of concurrent > updates, we might not have a stable opinion of which entry of the HOT > chain is live, leading to trying to index multiple ones of them (using > the same root TID), which leads t

Re: 13dev failed assert: comparetup_index_btree(): ItemPointer values should never be equal

2020-08-04 Thread Tom Lane
Peter Geoghegan writes: > On Tue, Aug 4, 2020 at 1:31 PM Tom Lane wrote: >> while the INSERT_IN_PROGRESS case has none. Simple symmetry >> would suggest that the INSERT_IN_PROGRESS case should be >> >> if (checking_uniqueness || >> HeapTupleIsHeapOnly(heapTuple)) >> // wa

Re: [patch] demote

2020-08-04 Thread Jehan-Guillaume de Rorthais
Hi, Yet another summary + patch + tests. Demote now keeps backends with no active xid alive. Smart mode keeps all backends: it waits for them to finish their xact and enter read-only. Fast mode terminate backends wit an active xid and keeps all other ones. Backends enters "read-only" using LocalX

Re: Confusing behavior of create table like

2020-08-04 Thread Tom Lane
Peter Eisentraut writes: > On 2020-08-04 19:36, Konstantin Knizhnik wrote: >> Yes, generated columns are also using implicitly generated sequences. >> So them are  very similar with SERIAL/BIGSERIAL columns. This actually >> make we wonder why we can not handle them in the same way in >> CREATE TA

Re: Concurrency bug in amcheck

2020-08-04 Thread Peter Geoghegan
Hi Alexander, On Tue, Aug 4, 2020 at 7:27 AM Alexander Korotkov wrote: > Thank you for your reminder. Revised patch is attached. Now, the contents > of deleted btree pages isn't masked. I've checked that installcheck passes > with wal_consistency_checking='Btree'. I'm going to push this if

Re: Concurrency bug in amcheck

2020-08-04 Thread Alexander Korotkov
On Wed, Aug 5, 2020 at 1:58 AM Peter Geoghegan wrote: > On Tue, Aug 4, 2020 at 7:27 AM Alexander Korotkov > wrote: > > Thank you for your reminder. Revised patch is attached. Now, the > > contents of deleted btree pages isn't masked. I've checked that > > installcheck passes with wal_consi

Re: Concurrency bug in amcheck

2020-08-04 Thread Peter Geoghegan
On Tue, Aug 4, 2020 at 4:18 PM Alexander Korotkov wrote: > Pushed. Comment is changed as you suggested. But I've replaced "last > pivot tuple" with "remaining tuples", because the page can also have a > high key, which is also a tuple. You're right, of course. Thanks again -- Peter Geoghegan

Re: LSM tree for Postgres

2020-08-04 Thread Alexander Korotkov
On Tue, Aug 4, 2020 at 7:56 PM Konstantin Knizhnik wrote: > I do not understand why do we need multiple indexes. > We need one "hot" index which fits in memory to perform fast inserts. > But it should not be too small to be able to accumulate substantial > amount of records to provide efficient bu

RE: Creating foreign key on partitioned table is too slow

2020-08-04 Thread kato-...@fujitsu.com
On Tuesday, July 14, 2020 11:29 PM, Daniel Fustafsson wrote: >Are you able to test Alvaros latest patch to see if that solves the originally >reported problem, so that we can reach >closure on this item during the >commitfest? Sorry for the too late replay. I missed this mail. And, thanks for w

Re: FailedAssertion("pd_idx == pinfo->nparts", File: "execPartition.c", Line: 1689)

2020-08-04 Thread Amit Langote
On Wed, Aug 5, 2020 at 9:52 AM Amit Langote wrote: > On Wed, Aug 5, 2020 at 9:32 AM Justin Pryzby wrote: > > On Wed, Aug 05, 2020 at 09:26:20AM +0900, Amit Langote wrote: > > > On Wed, Aug 5, 2020 at 12:11 AM Justin Pryzby > > > wrote: > > > > > > > > On Tue, Aug 04, 2020 at 08:12:10PM +0900, A

Re: Add MAIN_RELATION_CLEANUP and SECONDARY_RELATION_CLEANUP options to VACUUM

2020-08-04 Thread Bossart, Nathan
On 8/2/20, 11:47 PM, "Michael Paquier" wrote: > + VACOPT_TOAST_CLEANUP = 1 << 6, /* process TOAST table, if any */ > + VACOPT_DISABLE_PAGE_SKIPPING = 1 << 7, /* don't skip any pages */ > + VACOPT_MAIN_REL_CLEANUP = 1 << 8/* process main relation */ > } VacuumOption; > > Do we actually

Re: FailedAssertion("pd_idx == pinfo->nparts", File: "execPartition.c", Line: 1689)

2020-08-04 Thread Justin Pryzby
On Wed, Aug 05, 2020 at 09:53:44AM +0900, Amit Langote wrote: > On Wed, Aug 5, 2020 at 9:52 AM Amit Langote wrote: > > On Wed, Aug 5, 2020 at 9:32 AM Justin Pryzby wrote: > > > On Wed, Aug 05, 2020 at 09:26:20AM +0900, Amit Langote wrote: > > > > On Wed, Aug 5, 2020 at 12:11 AM Justin Pryzby >

Re: new heapcheck contrib module

2020-08-04 Thread Peter Geoghegan
On Tue, Aug 4, 2020 at 9:44 AM Robert Haas wrote: > I think it might be possible to distinguish between different types of > corruption and to separate, at least to some degree, the checking > associated with each type. I think one can imagine something that > checks the structure of a btree witho

Re: FailedAssertion("pd_idx == pinfo->nparts", File: "execPartition.c", Line: 1689)

2020-08-04 Thread Amit Langote
On Wed, Aug 5, 2020 at 10:04 AM Justin Pryzby wrote: > On Wed, Aug 05, 2020 at 09:53:44AM +0900, Amit Langote wrote: > > On Wed, Aug 5, 2020 at 9:52 AM Amit Langote wrote: > > > On Wed, Aug 5, 2020 at 9:32 AM Justin Pryzby wrote: > > > > On Wed, Aug 05, 2020 at 09:26:20AM +0900, Amit Langote wro

pg13dev: explain partial, parallel hashagg, and memory use

2020-08-04 Thread Justin Pryzby
I'm testing with a customer's data on pg13dev and got output for which Peak Memory doesn't look right/useful. I reproduced it on 565f16902. CREATE TABLE p(i int) PARTITION BY RANGE(i); CREATE TABLE p1 PARTITION OF p FOR VALUES FROM (0)TO(1000); CREATE TABLE p2 PARTITION OF p FOR VALUES FROM (1000

Re: pg13dev: explain partial, parallel hashagg, and memory use

2020-08-04 Thread David Rowley
On Wed, 5 Aug 2020 at 13:21, Justin Pryzby wrote: > > I'm testing with a customer's data on pg13dev and got output for which Peak > Memory doesn't look right/useful. I reproduced it on 565f16902. Likely the sanity of those results depends on whether you think that the Memory Usage reported outsi

Re: pg13dev: explain partial, parallel hashagg, and memory use

2020-08-04 Thread James Coleman
On Tue, Aug 4, 2020 at 9:44 PM David Rowley wrote: > > On Wed, 5 Aug 2020 at 13:21, Justin Pryzby wrote: > > > > I'm testing with a customer's data on pg13dev and got output for which Peak > > Memory doesn't look right/useful. I reproduced it on 565f16902. > > Likely the sanity of those results

Re: [DOC] Document concurrent index builds waiting on each other

2020-08-04 Thread Alvaro Herrera
On 2020-Mar-25, Andres Freund wrote: > What I was thinking of was a new flag, with a distinct value from > PROC_IN_VACUUM. It'd currently just be specified in the > GetCurrentVirtualXIDs() calls in WaitForOlderSnapshots(). That'd avoid > needing to wait for other CICs on different relations. Since

Re: pg13dev: explain partial, parallel hashagg, and memory use

2020-08-04 Thread Justin Pryzby
On Wed, Aug 05, 2020 at 01:44:17PM +1200, David Rowley wrote: > On Wed, 5 Aug 2020 at 13:21, Justin Pryzby wrote: > > > > I'm testing with a customer's data on pg13dev and got output for which Peak > > Memory doesn't look right/useful. I reproduced it on 565f16902. > > Likely the sanity of those

Re: [DOC] Document concurrent index builds waiting on each other

2020-08-04 Thread Alvaro Herrera
On 2020-Aug-04, Alvaro Herrera wrote: > diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h > index b20e2ad4f6..43c8ea3e31 100644 > --- a/src/include/storage/proc.h > +++ b/src/include/storage/proc.h > @@ -53,6 +53,8 @@ struct XidCache > #define PROC_IS_AUTOVACUUM

Re: pg13dev: explain partial, parallel hashagg, and memory use

2020-08-04 Thread David Rowley
On Wed, 5 Aug 2020 at 14:13, Justin Pryzby wrote: > Also odd (to me). If I encourage more workers, there are "slots" for each > "planned" worker, even though fewer were launched: Looking at explain.c for "num_workers; " (including the final space at the end), looking at each for loop that loops

Re: LSM tree for Postgres

2020-08-04 Thread Peter Geoghegan
On Tue, Aug 4, 2020 at 8:24 AM Alexander Korotkov wrote: > So, in my point of view LSM as an index AM is far not a full power LSM > for PostgreSQL, but it's still useful. Large insert-only tables can > benefit from LSM. Large tables with many indexes could also benefit, > because non-HOT updates

Re: pg13dev: explain partial, parallel hashagg, and memory use

2020-08-04 Thread David Rowley
On Wed, 5 Aug 2020 at 14:27, David Rowley wrote: > > On Wed, 5 Aug 2020 at 14:13, Justin Pryzby wrote: > > Also odd (to me). If I encourage more workers, there are "slots" for each > > "planned" worker, even though fewer were launched: > > Looking at explain.c for "num_workers; " (including the

Re: For standby pg_ctl doesn't wait for PM_STATUS_READY in presence of promote_trigger_file

2020-08-04 Thread Kyotaro Horiguchi
Hello. At Tue, 4 Aug 2020 12:01:45 -0700, Ashwin Agrawal wrote in > If shutdown (non hot enabled) standby and promote the standby using > promote_trigger_file via pg_ctl start with -w (wait), currently pg_ctl > returns as soon as recovery is started. Instead would be helpful if > pg_ctl can wai

Re: Handing off SLRU fsyncs to the checkpointer

2020-08-04 Thread Thomas Munro
On Tue, Aug 4, 2020 at 6:02 PM Thomas Munro wrote: > ... speedup of around 6% ... I did some better testing. OS: Linux, storage: consumer SSD. I repeatedly ran crash recovery on 3.3GB worth of WAL generated with 8M pgbench transactions. I tested 3 different builds 7 times each and used "minist

Reg. Postgres 13

2020-08-04 Thread Joel Mariadasan (jomariad)
Hi, I see that release 13 is currently in beta. When will be the official production release of 13 be out? We need to see if we can include this as part of our product release cycle. Regards, Joel

Re: LSM tree for Postgres

2020-08-04 Thread Konstantin Knizhnik
On 05.08.2020 02:59, Alexander Korotkov wrote: The things you're writing makes me uneasy. I initially understood lsm3 as a quick and dirty prototype, while you're probably keeping some design in your mind (for instance, original design of LSM). However, your message makes me think you're try