Re: Potential ABI breakage in upcoming minor releases

On Fri, Nov 15, 2024 at 11:39 PM Pavan Deolasee wrote: > > Looking more carefully at the usage of `ResultRelInfo` in the PGD code, I > think we might also be impacted by it. At one place, we loop through the > `es_result_relations` array and a size mismatch there will ca

Re: Potential ABI breakage in upcoming minor releases

On Fri, Nov 15, 2024 at 11:22 PM Noah Misch wrote: > > > > I'm starting to lean to the opinion that we need a re-wrap. > > Perhaps. Even if we do rewrap for some reason, it's not a given that > restoring the old struct size is net beneficial. If we restore the old > struct > size in v16.6, thos

Re: Potential ABI breakage in upcoming minor releases

On Fri, Nov 15, 2024 at 1:00 AM Tom Lane wrote: > Noah Misch writes: > > It's not immediately to clear to me why this would crash in a non-asserts > > build. palloc issues a 512-byte chunk for sizeof(ResultRelInfo)==368 on > v16, > > so I expect no actual writing past the end of the chunk. > >

Re: Potential ABI breakage in upcoming minor releases

On Thu, Nov 14, 2024 at 9:43 PM Peter Eisentraut wrote: > On 14.11.24 15:35, Noah Misch wrote: > > The postgr.es/c/e54a42a standard would have us stop here. But I'm open > to > > treating the standard as mistaken and changing things. > > That text explicitly calls out that adding struct members

Potential ABI breakage in upcoming minor releases

Hello, Commit 51ff46de29f67d73549b2858f57e77ada8513369 (backported all the way back to v12) added a new member to `ResultRelInfo` struct. This can potentially cause ABI breakage for the extensions that allocate the struct and pass it down to the PG code. The previously built extensions may allocat

Re: Backporting BackgroundPsql

H i Daniel, On Mon, Jul 1, 2024 at 1:09 PM Daniel Gustafsson wrote: > > On 29 Jun 2024, at 06:38, Pavan Deolasee > wrote: > > > Don't we need to add install and uninstall rules for the new module, > like we did in > https://git

Re: Backporting BackgroundPsql

On Thu, Jun 27, 2024 at 10:05 PM Heikki Linnakangas wrote: > > Ok, I pushed commits to backport BackgroundPsql down to v12. I used > "option 2", i.e. I changed background_psql() to return the new > BackgroundPsql object. > > Don't we need to add install and uninstall rules for the new module, lik

Question regarding ASSERT_NO_PARTITION_LOCKS_HELD_BY_ME in dshash_detach()

was very small before. If this is a problem worth addressing, I wonder if we should explicitly release all LWLocks in the long jump handler, like we do for other processes. Thanks, Pavan -- Pavan Deolasee EnterpriseDB: https://www.enterprisedb..com

Re: Assertion failure on PG15 with modified test_shm_mq test

ried if the code can be reached from multiple paths and testing all of those would be difficult for extension developers, especially given this may happen in error recovery path. Thanks, Pavan -- Pavan Deolasee EnterpriseDB: https://www.enterprisedb..com

Assertion failure on PG15 with modified test_shm_mq test

Hello, I've a slightly modified version of test_shm_mq, that I changed to include a shared fileset. The motivation to do that came because I hit an assertion failure with PG15 while doing some development work on BDR and I suspected it to be a PG15 bug. The stack trace looks as below: (lldb) bt

Re: Shmem queue is not flushed if receiver is not yet attached

e facility. Not sure how useful that is for your purpose. Thanks, Pavan -- Pavan Deolasee EnterpriseDB: https://www.enterprisedb..com

Re: Shmem queue is not flushed if receiver is not yet attached

le preserving the > essence of your proposed fix. > This looks good to me, > > I'm not sure that we need a shm_mq_flush(), but we definitely don't > have one currently, so I've also adjusted your patch to remove the > dead prototype. > > Makes sense to me. Tha

Shmem queue is not flushed if receiver is not yet attached

problem for me. On another note, `shm_mq.h` declares `shm_mq_flush()`, but I don't see it being implemented. Maybe just a leftover from the previous work? Though it seems useful to implement that API. Thanks, Pavan -- Pavan Deolasee EnterpriseDB: https://www.enterprisedb..com 0001-Flush-the-

Re: Assertion failure while streaming toasted data

ler value. But that same GUC is used to decide spilling txn to disk as well. So I am not sure if reducing the compile time default is acceptable or not. Thanks, Pavan -- Pavan Deolasee EnterpriseDB: https://www.enterprisedb..com

Re: Assertion failure while streaming toasted data

On Tue, May 25, 2021 at 6:55 PM Dilip Kumar wrote: > On Tue, May 25, 2021 at 6:50 PM Pavan Deolasee > wrote: > > > Is that test good enough to trigger the original bug? In my experience, > I had to add a lot more tuples before the logical_decoding_work_mem > threshol

Re: Assertion failure while streaming toasted data

kicked in. I would suggest running the test without the fix and check if the assertion hits. If so, we are good to go. Thanks, Pavan -- Pavan Deolasee EnterpriseDB: https://www.enterprisedb..com

Re: Assertion failure while streaming toasted data

em and run these tests with a much lower value. In fact, that's how I caught the problem in the first place. I had deliberately lowered the value to 1kB so that streaming code kicks in very often and even for small transactions. Thanks, Pavan -- Pavan Deolasee EnterpriseDB: https://www.enterprisedb..com

Re: Assertion failure while streaming toasted data

On Tue, May 25, 2021 at 1:49 PM Dilip Kumar wrote: > On Tue, May 25, 2021 at 1:45 PM Pavan Deolasee > wrote: > > > > I am not entirely sure if it works correctly. I'd tried something > similar, but the downstream node using > > my output plugin gets NULL value

Re: Assertion failure while streaming toasted data

lugin gets NULL values for the toast columns. It's a bit hard to demonstrate that with the test_decoding plugin, but if you have some other mechanism to test that change with an actual downstream node receiving and applying changes, it will be useful to test with that. Thanks, Pavan -- Pa

Assertion failure while streaming toasted data

nly triggers when data is inserted via COPY (multi-insert). Let me know if anything else is needed to reproduce this. Thanks, Pavan -- Pavan Deolasee EnterpriseDB: https://www.enterprisedb.com stream_toasted_txn_test.patch Description: Binary data

Re: some pointless HeapTupleHeaderIndicatesMovedPartitions calls

Hi Alvaro, On Tue, Sep 29, 2020 at 10:14 PM Alvaro Herrera wrote: > Hello > > Pavan Deolasee recently noted that a few of the > HeapTupleHeaderIndicatesMovedPartitions calls added by commit > 5db6df0c0117 are useless, since they are done after comparing t_self > with t_cti

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

gets into the tree. Thanks, Pavan -- Pavan Deolasee EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

pg_dump dumps row level policies on extension tables

or row level policies? Or somehow pg_dump should skip dumping those policies? Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

Hi Andres, On Wed, May 29, 2019 at 1:50 PM Pavan Deolasee wrote: > > > On Tue, 28 May 2019 at 4:36 PM, Andres Freund wrote: > >> Hi, >> >> On 2019-04-07 18:04:27 -0700, Andres Freund wrote: >> > Here's a *prototype* patch for this.

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

at others think before investing additional time. > > Pavan, are you planning to work on this for v13 CF1? Or have you lost > interest on the topic? > Yes, I plan to work on it. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: Caveats from reloption toast_tuple_target

he toast_tuple_target is specified at CREATE TABLE, but we would still have problem with ALTER TABLE, no? But there might be side effects of changing the lower limit for pg_dump/pg_restore. So we would need to think about that too. Thanks, Pavan -- Pavan Deolasee http://www.2nd

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

but not the vice versa. The proposed code does not introduce any new behaviour AFAICS. But I might be missing something. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

if major architectural changes are required then it's probably too late to consider this for PG12, even though it's more of a bug fix and a candidate for back-patching too. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: Re: A separate table level option to control compression

ERT INTO testtab VALUES ; Earlier this tuple would not have been toasted, even though toast_tuple_target is set to 256. But with the new code, the tuple will get toasted. So that's a change, but in the right direction as far as I can see. Also, this is a bit unrelated to what this patch is trying to achieve. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

On Wed, Mar 27, 2019 at 9:47 AM Masahiko Sawada wrote: > > The patch looks good to me. There is no comment from me. > > Thanks for your review! Updated patch attached since patch failed to apply after recent changes in the master. Thanks, Pavan -- Pavan Deolasee

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

on't see a harm in keeping the code as is. This code is borrowed from vacuumlazy.c and at some point we can even move it to some common location. > Perhaps we can add some tests for this feature to pg_visibility module. > > That's a good idea. Please see if the tests included i

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

in time to VACUUM the table. The benefits will only go up if the table is vacuumed much later when most of the pages are already written to the disk and removed from shared buffers and/or kernel cache. I hope this satisfies your doubts regarding performance implications of the patch.

Re: Re: A separate table level option to control compression

out negative effects of setting compress_tuple_target lower though, per your suggestion. Also added some details in storage.sgml as recommended by Sawada-san. Hope this helps. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services 0001-Add-a-table-level-option-to-control-compression.patch Description: Binary data

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

e scan that we are doing in this patch is done on a page that should be in the buffer cache, we will pay a bit in terms of CPU cost, but not anything in terms of IO cost. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

l-frozen. Since the check is performed immediately after the page becomes full and only once per page, there shouldn't be any additional IO cost and the check should be quite fast. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

intermediate blocks remain. I wonder if we should instead track the last used block in BulkInsertState and if the relcache invalidation flushes smgr, start inserting again from the last saved block. In fact, we already track the last used buffer in BulkInsertState and that's enough to

Re: [HACKERS] COPY FREEZE and PD_ALL_VISIBLE

d a new patch [1] for consideration to include in PG12. I started a new thread because the patch is completely new and this thread was a bit too old. Thanks, Pavan [1] https://www.postgresql.org/message-id/CABOikdN-ptGv0mZntrK2Q8OtfUuAjqaYMGmkdU1dCKFtUxVLrg%40mail.gmail.com -- Pavan Deolasee

COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

] https://www.postgresql.org/message-id/CAMkU%3D1w3osJJ2FneELhhNRLxfZitDgp9FPHee08NT2FQFmz_pQ%40mail.gmail.com -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services copy_freeze_v3.patch Description: Binary data

Re: Another way to fix inherited UPDATE/DELETE

parate mergeTargetRelation, but that approach has been criticised. May be Tom's idea doesn't have the same problem or most likely he will have a far better approach to address that. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

A separate table level option to control compression

RT will not compress the tuple since its length is less than the toast threshold. With the patch and after setting table level option, one can compress such tuples. The attached patch implements this idea. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL D

Re: MERGE SQL statement for PG12

if it wasn't last April, because nothing has changed since then. The purpose of keeping it up-to-date is to solicit feedback and directions and to show that my interest in the patch is still intact. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: monitoring CREATE INDEX [CONCURRENTLY]

OGRESS_CREATEIDX_PHASE 0 +/* 1 and 2 reserved for "waitfor" metrics */ +#define PROGRESS_CREATEIDX_PARTITIONS_TOTAL 3 +#define PROGRESS_CREATEIDX_PARTITIONS_DONE 4 + Is there a reason to leave those reserve placeholders, only to fill them a few lines down? Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: MERGE SQL statement for PG12

Hi Tomas, Sorry for a delayed response. On Mon, Oct 29, 2018 at 4:59 PM Tomas Vondra wrote: > Hi Pavan, > > On 10/29/2018 10:23 AM, Pavan Deolasee wrote: > > > > ... > > > > Thanks for keeping an eye on the patch. I've rebased the patch > > against t

Re: Accidental removal of a file causing various problems

relfilenode, a more interesting question is: should this be automatic or require administrative action? Does either of the ideas sound interesting enough for further work? Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: Accidental removal of a file causing various problems

On Sat, Aug 25, 2018 at 12:16 AM Tom Lane wrote: > Pavan Deolasee writes: > > 1. The user soon found out that they can no longer connect to any > database > > in the cluster. Not just the one to which the affected table belonged, > but > > no other database in the clu

Accidental removal of a file causing various problems

worth pursuing? Or are these side effects are well understood and known? IMHO even if we accept that we can't do much about a missing file, it seems quite odd that both 1 and 3 happens. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: PANIC during crash recovery of a recently promoted standby

e complete fix, verifying all the cases, in various back branches. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Locking considerations of REINDEX

be someday we would have ability so that the existing queries can continue to read from the old physical index, new queries will shift to the new index and eventually the old index's storage will be dropped when nobody can see it. Thanks, Pavan [1] https://www.postgresql.org/docs/11/static/sql-

Re: PANIC during crash recovery of a recently promoted standby

hat point. + */ s/that/this Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: PATCH: backtraces for error messages

t much more likely to get nowhere. > > Sorry, I did not mean to mix up two patches. I brought it up just in case it provides another idea about when and how to log the backtrace. So yeah, let's discuss that patch separately. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: PATCH: backtraces for error messages

tten and has a bad UI, but those can be improved if there is interest. Given the lack of response, I suspect there is enough interest in the feature though. Thanks, Pavan [1] https://www.postgresql.org/message-id/CABOikdMvx_Kr_b4ELhJEoeGcLTZKrDma%2BfPZpoZVdpL7Zc0bVw%40mail.gmail.com -- Pavan Deolas

Re: MERGE SQL statement for PG12

e complexity of the patch. So I thought it makes sense to submit it early. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: PANIC during crash recovery of a recently promoted standby

On Fri, May 11, 2018 at 8:39 PM, Alvaro Herrera wrote: > Michael Paquier wrote: > > On Thu, May 10, 2018 at 10:52:12AM +0530, Pavan Deolasee wrote: > > > I propose that we should always clear the minRecoveryPoint after > promotion > > > to ensure that crash recov

PANIC during crash recovery of a recently promoted standby

RecoveryPoint after promotion to ensure that crash recovery always run to the end if a just-promoted standby crashes before completing its first regular checkpoint. A WIP patch is attached. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Sup

Re: Toast issues with OldestXmin going backwards

How does this guard against the case when the same OID gets inserted in the toast table again, with matching lengths etc? Rare but seems possible, no? I think we should look for a more complete solution how hard these bugs are to detect and fix. Thanks, Pavan > -- > Andrew (irc:R

Re: VM map freeze corruption

ttached. Not fully polished (and there is a XXX that I left for comments), but hopefully enough to tell what I am thinking. Do you think it's any improvement or actually makes the problem worse? Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Developme

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

K blocks, remember those K buffers, discard those K buffers, truncate the relation and then try for next K blocks. If another backend requests lock on the table, we give up or retry after a while. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

heuristic based on available free space in the table prior to the truncation point? Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: VM map freeze corruption

pg_check_frozen('t');" | $p # See if a vacuum freeze scanning all pages corrects the problem echo "vacuum (verbose, freeze, disable_page_skipping) t;" | $p echo "select pg_check_frozen('t');" | $p Thanks, Pavan [1] https://www.postgresql.org/m

Re: Bugs in TOAST handling, OID assignment and redo recovery

On Thu, Apr 12, 2018 at 5:53 AM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 4/10/18 06:29, Pavan Deolasee wrote: > > One of our 2ndQuadrant support customers recently reported a sudden rush > > of TOAST errors post a crash recovery, nearly causing an ou

Re: Bugs in TOAST handling, OID assignment and redo recovery

s a very specialised test case written after getting a full grasp on the bug. And it just tests the thing that I knew is broken based on code reading. Also, with OID duplicate issue fixed, hitting more bugs in this area is going to be even more difficult. Thanks, Pavan -- Pavan Deolasee

Re: Bugs in TOAST handling, OID assignment and redo recovery

x27;t find a case that is broken. I even tried a few test cases with DDLs etc. But I think what you did is fine and more bullet proof. So +1 to that. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: Bugs in TOAST handling, OID assignment and redo recovery

On Wed, Apr 11, 2018 at 8:20 PM, Tom Lane wrote: > Pavan Deolasee writes: > > Or may be we simply err on the side of caution and scan the toast table > > with SnapshotAny while looking for a duplicate? That might prevent us > from > > reusing an OID for a known-dead tup

Re: Bugs in TOAST handling, OID assignment and redo recovery

On Tue, Apr 10, 2018 at 7:24 PM, Pavan Deolasee wrote: > Hi Heikki, > > On Tue, Apr 10, 2018 at 7:07 PM, Heikki Linnakangas > wrote: > >> >>> >> It would seem more straightforward to add a snapshot parameter to >> GetNewOidWithIndex(), so that th

Re: Faster inserts with mostly-monotonically increasing values

Yes, looks like an oversight :-( I will fix it along with the other changes that Peter requested. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: Bugs in TOAST handling, OID assignment and redo recovery

" tuples. > > Agreed. With nextXid, we advance ShmemVariableCache->nextXid if the value > in the online checkpoint record is greater than > ShmemVariableCache->nextXid. But we don't have such a wraparound-aware > concept of "greater than" for OIDs. I

Bugs in TOAST handling, OID assignment and redo recovery

gs exist there too. In fact, these bugs probably existed forever, though I did not check very old releases. Attached is a simple reproducer and a proposed fix to address both the bugs. We should consider backpatching it all supported releases. Thanks, Pavan -- Pavan Deolasee htt

Re: pgsql: New files for MERGE

th March: On Thu, Mar 29, 2018 at 3:20 PM, Simon Riggs wrote: > On 28 March 2018 at 12:00, Pavan Deolasee > wrote: > > > v27 attached, though review changes are in > > the add-on 0005 patch. > > This all looks good now, thanks for making all of those changes. >

Re: [HACKERS] MERGE SQL Statement for PG11

On Fri, Apr 6, 2018 at 1:30 AM, Andres Freund wrote: > Hi, > > On 2018-04-05 11:31:48 +0530, Pavan Deolasee wrote: > > > +/*- > > > > > > + * > > > + * nodeMerge.c > > >

Re: [HACKERS] MERGE SQL Statement for PG11

n move ExecInsert/Update etc to a new file as I suggested, but still use the ModifyTable to run Merge. There are many things common between them. ModifyTable executes all DMLs and MERGE is just another DML which can run all three. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

pile that way. We should either rewrite that assertion or put it inside a #ifdef ASSERT_CHECKING block or simple remove that assertion because we already check for relkind in parse_merge.c. Will check. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: Add support for printing/reading MergeAction nodes

more confident that we should have got the outfuncs.c support ok. debug_print_parse=on debug_print_rewritten=on debug_print_plan=on Also, I am now running tests with -DCOPY_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST since the buildfarm had earlier uncovered some issues with those flags. No p

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

ort ON CONFLICT DO UPDATE to move a row to a different partition, but otherwise it works now. See 555ee77a9668e3f1b03307055b5027e13bf1a715. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

* Do basic expression transformation (same as a > ROW() > +* expr, but allow SetToDefault at top level) > + */ > + exprList = transformExpressionList(pstate, > +

Re: pgsql: New files for MERGE

On Thu, Apr 5, 2018 at 12:16 AM, Andres Freund wrote: > Hi, > > On 2018-04-05 00:02:06 +0530, Pavan Deolasee wrote: > > Apologies from my end. Simon checked with me regarding your referenced > > email. I was in the middle of responding to it (with a add-on patch to > take

Re: pgsql: New files for MERGE

f your review comments), but got side tracked by some high priority customer escalation. I shall respond soon. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: Missing parse_merge.h?

to add new files in that commit. I am trying to get in touch with him so that he can add the missing files and correct the situation. Sorry for the trouble. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

uff which was necessitated when we added support for partitioned table. We discussed that at some length, with your inputs and agreed that it's not necessarily a bad thing and probably the only way to deal with partitioned tables. Personally, I don't see why an internal join is bad. That&

Re: [HACKERS] MERGE SQL Statement for PG11

TCHED processing, thus inserting it into the table again. I am not saying there is no scope for improvement. But we need to be careful about what can be pushed down to the join and what must be applied after the join. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] A design for amcheck heapam verification

e IndexBuildHeapScan() does that internally. I stumbled upon that while looking for any potential leaks. I know at least one other caller of IndexBuildHeapScan() doesn't bother to say anything either, but it's helpful. FWIW I also looked at the 0001 patch and it looks fine to me. Tha

Re: [HACKERS] A design for amcheck heapam verification

On Wed, Mar 28, 2018 at 2:48 AM, Peter Geoghegan wrote: > On Tue, Mar 27, 2018 at 6:48 AM, Pavan Deolasee > wrote: > > + * When index-to-heap verification is requested, a Bloom filter is used > to > > + * fingerprint all tuples in the target index, as the index is > tra

Re: [HACKERS] MERGE SQL Statement for PG11

On Wed, Mar 28, 2018 at 8:28 AM, Peter Geoghegan wrote: > On Tue, Mar 27, 2018 at 2:28 AM, Pavan Deolasee > wrote: > > (Version 26) > > I have some feedback on this version: > > * ExecMergeMatched() needs to determine tuple lock mode for > EvalPlanQual() in a way tha

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

what we do today. The idea is to just use a different mechanism to find the prior checkpoint. But we should surely find the latest prior checkpoint. In the rare scenario that Tom showed, we should just throw an error and fix the patch if it's not doing that already. Thanks, Pavan -- Pavan Deol

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

ase, we might read a WAL segment extra before we find the checkpoint record. That's not ideal but not too bad given that only pg_rewind needs this and that too only once. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] A design for amcheck heapam verification

olding OldestXmin back. Is there anything we can do to lessen that burden like telling other backends to ignore our xmin while computing OldestXmin (like vacuum does)? Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

On Tue, Mar 27, 2018 at 1:54 PM, Simon Riggs wrote: > On 26 March 2018 at 17:06, Simon Riggs wrote: > > On 26 March 2018 at 15:39, Pavan Deolasee > wrote: > > > > > That's all I can see so far. > > * change comment “once to” to “once” in src/include/nodes/

Re: PATCH: Exclude unlogged tables from base backups

On Mon, Mar 26, 2018 at 5:16 PM, Masahiko Sawada wrote: > On Mon, Mar 26, 2018 at 4:52 PM, Pavan Deolasee > > >> > > > > This one-liner patch fixes it for me. > > > > Isn't this issue already fixed by commit > d0c0c894533f906b13b79813f02b2982ac6

Re: PATCH: Exclude unlogged tables from base backups

On Mon, Mar 26, 2018 at 1:03 PM, Pavan Deolasee wrote: > On Fri, Mar 23, 2018 at 9:51 PM, David Steele wrote: > >> On 3/23/18 12:14 PM, Teodor Sigaev wrote: >> > >> > Thank you, pushed >> >> > Is it just me or the newly added test in 010_pg_basebacku

Re: Faster inserts with mostly-monotonically increasing values

On Sun, Mar 25, 2018 at 6:00 AM, Andrew Dunstan < andrew.duns...@2ndquadrant.com> wrote: > On Fri, Mar 23, 2018 at 8:27 PM, Pavan Deolasee > wrote: > >> > >> > >> I would probably just have a few regression lines that should be sure > >>

Re: PATCH: Exclude unlogged tables from base backups

; and thus fails the test. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

but it would > at least alleviate the spinlock. This gives us the same level of guarantee that xl_prev used to offer, yet help us use atomic operations. I'll be happy if we can look at that particular change and see if there are any holes there. Thanks, Pavan -- Pavan Deolasee

Re: [HACKERS] MERGE SQL Statement for PG11

hat the CTE itself is assigned plan_id 1 and the SubPlan then gets plan_id 2. I can investigate further, but given that we see a similar behaviour with regular UPDATE, I don't think it's worth. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

On Fri, Mar 23, 2018 at 4:43 AM, Peter Geoghegan wrote: > On Thu, Mar 22, 2018 at 11:42 AM, Pavan Deolasee > wrote: > > A slightly improved version attached. > > You still need to remove this change: > > > diff --git a/src/include/miscadmin.h b/src/include/misca

Re: [HACKERS] MERGE SQL Statement for PG11

is because it increases my overall > confidence in the design. If it was genuinely hard to add WITH clause > support, then that would probably tell us something about the overall > design that likely creates problems elsewhere. It's easy to say that > it isn't worth hold

Re: Faster inserts with mostly-monotonically increasing values

a few regression lines that should be sure > to exercise the code path and leave it at that. > > I changed the regression tests to include a few more scenarios, basically using multi-column indexes in different ways and they querying rows by ordering rows in different ways. I did not take aw

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

Master: == #clients #tps 1 24.128004 2 12.326135 4 8.334143 8 16.035699 16 8.502794 So that's pretty good improvement across the spectrum. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services pg_reduce_wal_contention_v2.patch Description: Binary data

Re: [HACKERS] MERGE SQL Statement for PG11

On Fri, Mar 23, 2018 at 10:00 AM, Amit Langote < langote_amit...@lab.ntt.co.jp> wrote: > On 2018/03/23 3:42, Pavan Deolasee wrote: > > A slightly improved version attached. Apart from doc cleanup based on > > earlier feedback, fixed one assertion failure based on Rahila&#

Re: [HACKERS] MERGE SQL Statement for PG11

ROM pg_class), (SELECT relname FROM pg_class LIMIT 1)); MERGE 1 postgres=# SELECT * FROM target; a | b -+ 755 | pgbench_source (1 row) Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: ON CONFLICT DO UPDATE for partitioned tables

> ResultRelInfo as I wrote above. > > Thanks. It's looking much better now. I think we can possibly move all ON CONFLICT related members to a separate structure and just copy the pointer to the structure if (map == NULL). That might make the code a bit more tidy. Is there anything that needs to be done for transition tables? I checked and didn't see anything, but please check. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: Faster inserts with mostly-monotonically increasing values

dated patch. I removed the last offset caching thing completely and integrated your changes for conditional lock access. Some other improvements to test cases too. I realised that we must truncate and re-insert to test index fastpath correctly. Thanks, Pavan -- Pavan Deolasee

  1   2   >