Re: Single transaction in the tablesync worker?

2020-12-21 Thread Peter Smith
xcessive logging of mine scattered around which I added to help my testing during development * Address review comments --- Kind Regards, Peter Smith. Fujitsu Australia v5-0001-2PC-change-tablesync-slot-to-use-same-two_phase-m.patch Description: Binary data v5-0002-WIP-patch-for-the-Solution1.p

Re: Single transaction in the tablesync worker?

2020-12-21 Thread Peter Smith
On Sat, Dec 19, 2020 at 5:38 PM Amit Kapila wrote: > > On Fri, Dec 18, 2020 at 6:41 PM Peter Smith wrote: > > > > TODO / Known Issues: > > > > * the current implementation of tablesync drop slot (e.g. from > > DropSubscription or finish_sync_worker) regenerat

Re: Single transaction in the tablesync worker?

2020-12-21 Thread Peter Smith
sync slot. > + */ > + { > + extern void ReplicationSlotDropAtPubNode( > + WalReceiverConn *wrconn_given, char *conninfo, char *subname, char > *slotname); > > This is not how we export functions at other places? Fixed in latest v5 patch - https://www.postgresql.org/message-id/CAHut%2BPvmDJ_EO11_up%3D_cRbOjhdWCMG-n7kF-mdRhjtCHcjHRA%40mail.gmail.com Kind Regards, Peter Smith. Fujitsu Australia.

Re: Single transaction in the tablesync worker?

2020-12-22 Thread Peter Smith
d around which I added to help my testing during development * Address review comments --- Kind Regards, Peter Smith. Fujitsu Australia v6-0002-WIP-patch-for-the-Solution1.patch Description: Binary data v6-0001-2PC-change-tablesync-slot-to-use-same-two_phase-m.patch Description: Binary data

Re: Single transaction in the tablesync worker?

2020-12-22 Thread Peter Smith
On Mon, Dec 21, 2020 at 11:36 PM Amit Kapila wrote: > > On Mon, Dec 21, 2020 at 3:17 PM Peter Smith wrote: > > > > On Mon, Dec 21, 2020 at 4:23 PM Amit Kapila wrote: > > > > > Few other comments: > > > == > > > > Thanks for

HOT chain bug in latestRemovedXid calculation

2020-12-22 Thread Peter Geoghegan
2. heap_compute_xid_horizon_for_tuples() is a descendant of code added by Simon’s commit a760893d in 2010 -- pretty close to HOT’s initial introduction. So this has been around for a long time. -- Peter Geoghegan 0001-Instrument-heap_compute_xid_horizon_for_tuples.patch Description: Binary data

Re: Single transaction in the tablesync worker?

2020-12-22 Thread Peter Smith
temporary "!!>>" excessive logging scattered around which I added to help my testing during development * Address review comments --- Kind Regards, Peter Smith. Fujitsu Australia v7-0002-WIP-patch-for-the-Solution1.patch Description: Binary data v7-0001-2PC-change-tablesync-

Re: Single transaction in the tablesync worker?

2020-12-23 Thread Peter Smith
v6 patch. TODO / Known Issues: * Help / comments * There is temporary "!!>>" excessive logging scattered around which I added to help my testing during development * Address review comments --- Kind Regards, Peter Smith. Fujitsu Australia v8-0001-WIP-patch-for-the-Solution1.p

Re: New IndexAM API controlling index vacuum strategies

2020-12-23 Thread Peter Geoghegan
zy vacuum to not do what gets requested by amvacuumcleanup() when it cannot respect the wishes of one individual indexes, for example when the accumulation of LP_DEAD items in the heap becomes a big problem in itself? That really could be the thing that forces full heap vacuuming, even with several indexes. I will need to experiment in order to improve my understanding of how to make this cooperate with bottom-up index deletion. But that's mostly just a question for my patch (and a relatively easy one). -- Peter Geoghegan

vacuum_cost_page_miss default value and modern hardware

2020-12-27 Thread Peter Geoghegan
e. I expect little impact on small installations. [1] https://postgr.es/m/cad21aod0ske11fmw4jd4renawbmcw1wasvnwpjvw3tvqpoq...@mail.gmail.com -- Peter Geoghegan

Re: New IndexAM API controlling index vacuum strategies

2020-12-27 Thread Peter Geoghegan
ept in the extreme case where the index is *exactly* the same size as it was after the last VACUUM. This will happen regularly with bottom-up index deletion. Maybe that approach is a bit too conservative, though. -- Peter Geoghegan

Re: pgsql: Fix memory leak in plpgsql's CALL processing.

2020-12-28 Thread Peter Eisentraut
This change has broken the following test case (reduced from actual production code): CREATE OR REPLACE PROCEDURE prc_inout_test_inner(INOUT x integer) LANGUAGE plpgsql AS $procedure$ BEGIN COMMIT; x := 2; END; $procedure$; CREATE OR REPLACE PROCEDURE prc_inout_test_main() LANGUAGE plpgsql

trailing junk in numeric literals

2020-12-28 Thread Peter Eisentraut
I was surprised to find that this doesn't error: => select 100a; a - 100 I suspect this and similar cases used to error before aliases without AS were introduced. But now this seems possibly problematic. Should we try to handle this better?

Re: New IndexAM API controlling index vacuum strategies

2020-12-28 Thread Peter Geoghegan
On Sun, Dec 27, 2020 at 11:41 PM Peter Geoghegan wrote: > I experimented with this today, and I think that it is a good way to > do it. I like the idea of choose_vacuum_strategy() understanding that > heap pages that are subject to many non-HOT updates have a "natural > extra cap

Re: HOT chain bug in latestRemovedXid calculation

2020-12-28 Thread Peter Geoghegan
On Tue, Dec 22, 2020 at 9:52 AM Peter Geoghegan wrote: > ISTM that heap_compute_xid_horizon_for_tuples() calculates > latestRemovedXid for index deletion callers without sufficient care. > The function only follows line pointer redirects, which is necessary > but not sufficient

Re: New IndexAM API controlling index vacuum strategies

2020-12-28 Thread Peter Geoghegan
actually happen, but it's possible in theory, and if it did happen it really wouldn't be so bad). [1] https://postgr.es/m/CAH2-Wz=rpkb5vxs7az+v8t3ve75v_dkgro+w4nwd64e9yic...@mail.gmail.com -- Peter Geoghegan

Re: New IndexAM API controlling index vacuum strategies

2020-12-28 Thread Peter Geoghegan
On Mon, Dec 28, 2020 at 11:20 PM Peter Geoghegan wrote: > > That's a very good result in terms of skipping lazy_vacuum_heap(). How > > much the table and indexes bloated? Also, I'm curious about that which > > tests in choose_vacuum_strategy() turned vacuum_heap

Re: trailing junk in numeric literals

2020-12-29 Thread Peter Eisentraut
On 2020-12-28 21:54, Tom Lane wrote: Peter Eisentraut writes: I was surprised to find that this doesn't error: => select 100a; a - 100 I suspect this and similar cases used to error before aliases without AS were introduced. But now this seems possibly problematic. S

Re: Single transaction in the tablesync worker?

2020-12-29 Thread Peter Smith
ring development * Address review comments --- Kind Regards, Peter Smith. Fujitsu Australia v9-0001-WIP-patch-for-the-Solution1.patch Description: Binary data

Re: Single transaction in the tablesync worker?

2020-12-29 Thread Peter Smith
On Wed, Dec 23, 2020 at 9:07 PM Amit Kapila wrote: > > On Tue, Dec 22, 2020 at 4:58 PM Peter Smith wrote: > > > > On Mon, Dec 21, 2020 at 11:36 PM Amit Kapila > > wrote: > > > > > > On Mon, Dec 21, 2020 at 3:17 PM Peter Smith wrote: > > > &g

Re: Single transaction in the tablesync worker?

2020-12-29 Thread Peter Smith
ly, consider if we error out after the second > transaction, we won't where to start decoding from. I think all these > should be done in a single transaction. Fixed as suggested. Please see latest patch [v9] --- [v9] https://www.postgresql.org/message-id/CAHut%2BPv8ShLmrjCriVU%2Btprk_9b2kwBxYK2oGSn5Eb__kWVc7A%40mail.gmail.com Kind Regards, Peter Smith. Fujitsu Australia

Re: dynamic result sets support in extended query protocol

2020-12-30 Thread Peter Eisentraut
]: https://www.postgresql.org/message-id/4f733cca-5e07-e167-8b38-05b5c9066d04%402ndQuadrant.com -- Peter Eisentraut 2ndQuadrant, an EDB company https://www.2ndquadrant.com/ From 03e7b009ca54f686f0798c764ffc802e70f64076 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Wed, 30 Dec 2020 14:3

Re: HOT chain bug in latestRemovedXid calculation

2020-12-30 Thread Peter Geoghegan
On Mon, Dec 28, 2020 at 9:49 PM Peter Geoghegan wrote: > I would like to commit this patch to v12, the first version that did > this process during original execution rather than in REDO routines. > It seems worth keeping the back branches in sync here. I suspect that > the old a

Re: Deleting older versions in unique indexes to avoid page splits

2020-12-31 Thread Peter Geoghegan
UP_FAVORABLE_STRIDE 3 > > Adding a comment on why the number 3 is chosen would be helpful for people to > understand the code. Noted - I will add something about that to the next revision. -- Peter Geoghegan

Re: Why latestRemovedXid|cuteoff_xid are always sent?

2021-01-02 Thread Peter Geoghegan
without scanning the proc array (in cases where it will still do so today, since of course ResolveRecoveryConflictWithSnapshot() has the obvious InvalidTransactionId fast path already). > Is it a sane idea or I have missed something huge? It seems like two almost distinct ideas to me. Thoug

pg_waldump/heapdesc.c and struct field names

2021-01-03 Thread Peter Geoghegan
master, so that pg_waldump will use struct field names? It doesn't seem necessary to change the output that has spaces instead of an underscore, or something like that. It just seems worth removing gratuitous inconsistencies, such as this one. -- Peter Geoghegan

Re: Single transaction in the tablesync worker?

2021-01-04 Thread Peter Smith
ments --- Kind Regards, Peter Smith. Fujitsu Australia v10-0002-Tablesync-extra-logging.patch Description: Binary data v10-0001-Tablesync-Solution1.patch Description: Binary data

Re: Single transaction in the tablesync worker?

2021-01-04 Thread Peter Smith
se to separate patch. Fixed in latest patch (v10). > > 2. Remove WIP from the commit message and patch name. > > -- Fixed in latest patch (v10) --- v10 = https://www.postgresql.org/message-id/CAHut%2BPuzPmFzk3p4oL9H3nkiY6utFryV9c5dW6kRhCe_RY%3DgnA%40mail.gmail.com Kind Regards, Peter Smith. Fujitsu Australia

Re: Context diffs

2021-01-04 Thread Peter Geoghegan
big issue for me. I really find it convenient to use "git format-patch" to produce patches for complicated long lived projects -- having commit messages and explicit ordering of multiple patches is really helpful. I've come to prefer unified, perhaps as a result of using "git format-patch" so much. -- Peter Geoghegan

Re: pg_waldump/heapdesc.c and struct field names

2021-01-04 Thread Peter Geoghegan
s as a symbol name 84 times, across many files in several distinct areas. Whereas remxid means nothing to me unless I go look it up, which is not ideal. -- Peter Geoghegan

Re: pg_waldump/heapdesc.c and struct field names

2021-01-04 Thread Peter Geoghegan
On Sun, Jan 3, 2021 at 8:58 PM Masahiko Sawada wrote: > On Mon, Jan 4, 2021 at 12:55 PM Peter Geoghegan wrote: > +1 for changing heapdesc.c on master. It's not only readable but also > consistent with other *desc showing the field named latestRemovedXid. > For instan

Fix typo in comment

2021-01-04 Thread Peter Smith
PSA a trivial patch to correct a comment typo. Kind Regards, Peter Smith Fujitsu Australia. fix_typo.patch Description: Binary data

Adding new commitfest entry?

2021-01-04 Thread Peter Smith
Hi. I wanted to add a new commitfest entry but although I am logged in I do not see any "Add" button either in the current or future CF. https://commitfest.postgresql.org/31/ https://commitfest.postgresql.org/32/ How to add? Kind Regards, Peter Smith. Fujitsu Australia.

Re: Adding new commitfest entry?

2021-01-04 Thread Peter Smith
On Tue, Jan 5, 2021 at 11:36 AM Tom Lane wrote: > Hm, there should be a "New patch" button just below the "Commitfest > 2021-03" title. Not for me. I see only 2 buttons - "Search/Filter" and "Shortcuts". --- Kind Regards, Peter Smith. Fujitsu Australia.

Re: Adding new commitfest entry?

2021-01-04 Thread Peter Smith
On Tue, Jan 5, 2021 at 12:46 PM Masahiko Sawada wrote: > > On Tue, Jan 5, 2021 at 9:45 AM Tom Lane wrote: > > > > Peter Smith writes: > > > On Tue, Jan 5, 2021 at 11:36 AM Tom Lane wrote: > > >> Hm, there should be a "New patch" button

Re: pg_waldump/heapdesc.c and struct field names

2021-01-04 Thread Peter Geoghegan
On Mon, Jan 4, 2021 at 2:06 PM Peter Geoghegan wrote: > Right. Self-consistency matters, as does consistency with the source > code itself. Pushed a commit that standardizes on the name latestRemovedXid within rmgr desc output routines just now. Thanks -- Peter Geoghegan

Re: Single transaction in the tablesync worker?

2021-01-05 Thread Peter Smith
tracking is cleaned up during DropSubscription and/or process_syncing_tables_for_apply. * the DropSubscription cleanup code was enhanced (v7+) to take care of crashed sync workers. * minor updates to PG docs TODO / Known Issues: * address review comments --- Kind Regards, Peter Smith. Fujitsu

Re: Single transaction in the tablesync worker?

2021-01-05 Thread Peter Smith
ginId node, > LWLockAcquire(&replication_state->lock, LW_EXCLUSIVE); > > /* Make sure it's not used by somebody else */ > - if (replication_state->acquired_by != 0) > + if (replication_state->acquired_by != 0 && > replication_state->acquired_by != MyProcPid) > { > TODO > I think you won't need this change if you do replorigin_advance before > replorigin_session_setup in your patch. > > 8. > - * that ensures we won't loose knowledge about that after a crash if the > + * that ensures we won't lose knowledge about that after a crash if the > > It is better to submit this as a separate patch. > Done. Please see CF entry. https://commitfest.postgresql.org/32/2926/ [v11] = https://www.postgresql.org/message-id/CAHut%2BPu0A6TUPgYC-L3BKYQfa_ScL31kOV_3RsB3ActdkL1iBQ%40mail.gmail.com Kind Regards, Peter Smith. Fujitsu Australia.

Re: [PATCH] Runtime control of CLOBBER_CACHE_ALWAYS

2021-01-05 Thread Peter Eisentraut
aches_always=N. Thoughts? -- Peter Eisentraut 2ndQuadrant, an EDB company https://www.2ndquadrant.com/

Re: Deleting older versions in unique indexes to avoid page splits

2021-01-05 Thread Peter Geoghegan
duplicates in indexes". I'll add something like that to the next revision. > I see. Would be good to explain that pattern with multiple indexes in > the comment. Will do -- it is the single best example of how heap block locality can matter with a real workload, so it makes sense to go with it in explanatory comments. -- Peter Geoghegan

Re: Single transaction in the tablesync worker?

2021-01-05 Thread Peter Smith
e = NONE) workaround becomes ineffectual. Note - the current patch code is only logging when the user has already disassociated the slot name; of course normally (when the slot name was not disassociated) table slots will give ERROR for broken connections. IMO, if the user has disassociated the slot name then they have already made their decision that they REALLY DO want to “proceed in this situation”. So I thought we should let them proceed. What do you think? Kind Regards, Peter Smith. Fujitsu Australia.

Re: Single transaction in the tablesync worker?

2021-01-06 Thread Peter Smith
On Wed, Jan 6, 2021 at 4:04 PM Amit Kapila wrote: > > On Tue, Jan 5, 2021 at 3:32 PM Peter Smith wrote: > > > > On Mon, Jan 4, 2021 at 10:48 PM Amit Kapila wrote: > > > > > > > > 5. Is it possible to write a testcase where we fail (say due to pk >

Re: [PATCH] Runtime control of CLOBBER_CACHE_ALWAYS

2021-01-06 Thread Peter Eisentraut
I don't think that necessarily works for all settings. Maybe we should make a pass through it and ensure it all works sensibly, but that seems like a separate project. -- Peter Eisentraut 2ndQuadrant, an EDB company https://www.2ndquadrant.com/

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-06 Thread Peter Geoghegan
speak of 15x. +1. I am *strongly* opposed to enabling checksums by default for this reason. I think that it's a total non-starter, unless and until the overhead can be dramatically reduced. The fact that it isn't the fault of the checksum mechanism in some abstract sense is 100% irrelevant. -- Peter Geoghegan

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-06 Thread Peter Geoghegan
d this argument baffling. Do you really believe this? -- Peter Geoghegan

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-06 Thread Peter Geoghegan
and that it logically follows that disagreeing with you is essentially irresponsible. This is a tactic that would be an embarrassment to a high school debate team. It's below you. Your argument may be logically consistent, but it's still nonsense. -- Peter Geoghegan

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-06 Thread Peter Geoghegan
appens. We added page-level checksums in 9.3. Can you imagine a counterfactual history in which Postgres had page checksums since the 1990s, but only added the fsync feature in 9.3? Please answer this non-rhetorical question. -- Peter Geoghegan

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-06 Thread Peter Geoghegan
ing "show data_checksums;" in psql in each case? What does "show full_page_writes;" show you? -- Peter Geoghegan

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-06 Thread Peter Geoghegan
On Wed, Jan 6, 2021 at 1:08 PM Peter Geoghegan wrote: > So you tested this using "show data_checksums;" in psql in each case? > > What does "show full_page_writes;" show you? Another consideration is checkpoint_timeout and max_wal_size. -- Peter Geoghegan

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-06 Thread Peter Geoghegan
e to say on this thread. I am -1 on the current proposal to enable page-level checksums by default. I may change my mind on this in the future, perhaps when underlying architectural details change, but right now this seems like a net negative. -- Peter Geoghegan

Re: vacuum_cost_page_miss default value and modern hardware

2021-01-06 Thread Peter Geoghegan
irtying the page) should be cheaper than dirtying a page. This thread began because I wanted to discuss the relative cost of different kinds of I/O operations to VACUUM, without necessarily discussing the absolute costs/time delays. -- Peter Geoghegan

Re: vacuum_cost_page_miss default value and modern hardware

2021-01-06 Thread Peter Geoghegan
On Wed, Jan 6, 2021 at 5:39 PM Masahiko Sawada wrote: > > On Mon, Dec 28, 2020 at 5:17 AM Peter Geoghegan wrote: > > > > Simply decreasing vacuum_cost_page_dirty seems like a low risk way of > > making the VACUUM costing more useful within autovacuum workers. > >

Re: Single transaction in the tablesync worker?

2021-01-06 Thread Peter Smith
ifferent from the macro define. > Yes, same was already previously reported [1] [1] https://www.postgresql.org/message-id/CAA4eK1Kyi037XZzyrLE71MS2KoMmNSqa6RrQLdSCeeL27gnL%2BA%40mail.gmail.com It will be fixed in the next patch (v12) Kind Regards, Peter Smith. Fujitsu Australia.

Re: Single transaction in the tablesync worker?

2021-01-06 Thread Peter Smith
DropSubscription cleanup code was enhanced (v7+) to take care of any crashed tablesync workers. * Updates to PG docs TODO / Known Issues: * Address review comments * Patch applies with whitespace warning --- Kind Regards, Peter Smith. Fujitsu Australia v12-0001-Tablesync-Solution1.patch Description

Re: Single transaction in the tablesync worker?

2021-01-07 Thread Peter Smith
On Mon, Jan 4, 2021 at 8:06 PM Amit Kapila wrote: > > On Wed, Dec 30, 2020 at 11:51 AM Peter Smith wrote: > > > > On Wed, Dec 23, 2020 at 8:43 PM Amit Kapila wrote: > > > > > > 1. > > > + * Rarely, the DropSubscription may be issued when a tablesync

Re: Single transaction in the tablesync worker?

2021-01-07 Thread Peter Smith
gml > @@ -7651,6 +7651,7 @@ SCRAM-SHA-256$<iteration > count>:&l > State code: > i = initialize, > d = data is being copied, > + C = table data has been copied, > s = synchronized, > Fixed in latest patch [v12] [v12] = https://www.postgresql.org/message-id/CAHut%2BPsonJzarxSBWkOM%3DMjoEpaq53ShBJoTT9LHJskwP3OvZA%40mail.gmail.com Kind Regards, Peter Smith. Fujitsu Australia

Re: Single transaction in the tablesync worker?

2021-01-07 Thread Peter Smith
/devel/logical-replication-subscription.html > PG docs updated in latest patch [v12] [v12] = https://www.postgresql.org/message-id/CAHut%2BPsonJzarxSBWkOM%3DMjoEpaq53ShBJoTT9LHJskwP3OvZA%40mail.gmail.com Kind Regards, Peter Smith. Fujitsu Australia

Re: Prevent printing "next step instructions" in initdb and pg_upgrade

2021-01-07 Thread Peter Eisentraut
on is right now.) -- Peter Eisentraut 2ndQuadrant, an EDB company https://www.2ndquadrant.com/

Re: Why latestRemovedXid|cuteoff_xid are always sent?

2021-01-07 Thread Peter Geoghegan
On Sat, Jan 2, 2021 at 3:22 PM Peter Geoghegan wrote: > Of course, it's possible that the question of whether or not it's > worth it has been misjudged for any given case. And maybe these > particular WAL records are one such case where somebody got it wrong, > affecting

Re: data_checksums enabled by default (was: Move --data-checksums to common options in initdb --help)

2021-01-07 Thread Peter Geoghegan
with this analysis, but since you only seem to want to discuss the question in a narrow way (e.g. "I agree that improving compression performance would be good but I don't see that as relevant to the question of what our defaults should be"), I have to wonder if it's worth the trouble. -- Peter Geoghegan

Re: Deleting older versions in unique indexes to avoid page splits

2021-01-07 Thread Peter Geoghegan
hould "figure out the best strategy for handling bloat on its own". A diversity of strategies are available, which can adapt to many different kinds of workloads organically. That kind of approach is sometimes necessary with a complex system IMV. [1] https://btw.informatik.uni-rostock.de/download/tagungsband/B2-2.pdf -- Peter Geoghegan

Re: Single transaction in the tablesync worker?

2021-01-07 Thread Peter Smith
On Thu, Jan 7, 2021 at 3:20 PM Amit Kapila wrote: > > On Wed, Jan 6, 2021 at 3:39 PM Amit Kapila wrote: > > > > On Wed, Jan 6, 2021 at 2:13 PM Peter Smith wrote: > > > > > > I think it makes sense. If there can be a race between the tablesync >

Re: Single transaction in the tablesync worker?

2021-01-08 Thread Peter Smith
. See FIXME comments. --- Kind Regards, Peter Smith. Fujitsu Australia On Thu, Jan 7, 2021 at 6:52 PM Peter Smith wrote: > > Hi Amit. > > PSA the v12 patch for the Tablesync Solution1. > > Differences from v11: > + Added PG docs to mention the tablesync slot > + R

Re: Improper use about DatumGetInt32

2021-01-08 Thread Peter Eisentraut
On 2020-12-25 08:45, Michael Paquier wrote: If we really think that we ought to differentiate, then we could do what pg_stat_statement does, and have a separate C function that's called with the obsolete signature (pg_stat_statements_1_8 et al). With the 1.8 flavor, it is possible to pass down a

Re: Single transaction in the tablesync worker?

2021-01-08 Thread Peter Smith
id, albeit maybe unnecessary since in the current code the tablesync slot name length is fixed. But I left the older comment here as a safety reminder in case some future change would want to modify the slot name. What do you think? [v13] = https://www.postgresql.org/message-id/CAHut%2BPvby4zg6kM1RoGd_j-xs9OtPqZPPVhbiC53gCCRWdNSrw%40mail.gmail.com Kind Regards, Peter Smith. Fujitsu Australia.

Re: Improper use about DatumGetInt32

2021-01-08 Thread Peter Eisentraut
On 2021-01-08 10:21, Peter Eisentraut wrote: I think on 64-bit systems it's actually safe, but on 32-bit systems (with USE_FLOAT8_BYVAL), if you use the new binaries with the old SQL-level definitions, you'd get the int4 that is passed in interpreted as a pointer, which would lead t

Re: Improper use about DatumGetInt32

2021-01-09 Thread Peter Eisentraut
On 2021-01-09 02:46, Michael Paquier wrote: +/* LCOV_EXCL_START */ Does it really make sense to add those markers here? It seems to me that we would ignore any new coverage if regression tests based on older versions are added (we may really want to have such tests for more in-core extensions to

snowball update

2021-02-12 Thread Peter Eisentraut
There is another snowball release out, and I have prepared a patch to integrate it. It's very big and mostly boring, so I'm not attaching it here, but you can see it at https://github.com/petere/postgresql/commit/d0aa6c2148bcef10942959035ce14f1810873593.patch Major changes are new stemmers fo

Re: PostgreSQL <-> Babelfish integration

2021-02-12 Thread Peter Geoghegan
ode available. Maybe that will change tomorrow or next week, but as of this moment there is simply nothing substantive to evaluate. -- Peter Geoghegan

Re: PostgreSQL <-> Babelfish integration

2021-02-12 Thread Peter Geoghegan
res development are not set in any fixed way (except to the limited extent that you're on the hook for anything you integrate that breaks). I myself am not convinced that this is worth spending any time on right now, especially given the lack of code to evaluate. -- Peter Geoghegan

Re: PostgreSQL <-> Babelfish integration

2021-02-12 Thread Peter Geoghegan
be downloaded and tested. -- Peter Geoghegan

Re: 64-bit XIDs in deleted nbtree pages

2021-02-12 Thread Peter Geoghegan
't take very long for a deleted page to become recyclable -- why wait? This idea is enabled by commit c79f6df75dd from 2018. I think it's the next logical step. -- Peter Geoghegan

Re: 64-bit XIDs in deleted nbtree pages

2021-02-13 Thread Peter Geoghegan
e specific scenario that Masahiko expressed concern about. Nor does it seem relevant to this patch more generally. -- Peter Geoghegan

Re: 64-bit XIDs in deleted nbtree pages

2021-02-13 Thread Peter Geoghegan
On Fri, Feb 12, 2021 at 9:04 PM Peter Geoghegan wrote: > On Fri, Feb 12, 2021 at 8:38 PM Masahiko Sawada wrote: > > I agree that there already are huge problems in that case. But I think > > we need to consider an append-only case as well; after bulk deletion > > on an appe

Re: GlobalVisIsRemovableFullXid() vs GlobalVisCheckRemovableXid()

2021-02-14 Thread Peter Geoghegan
() actually be called ReadNewFullTransactionId()? Actually, the inverse approach looks like it produces fewer inconsistencies: you could instead rename ReadNewTransactionId() to ReadNextTransactionId(). -- Peter Geoghegan

Re: GlobalVisIsRemovableFullXid() vs GlobalVisCheckRemovableXid()

2021-02-14 Thread Peter Geoghegan
that it's slightly strange that the 32 and 64 bit versions differ > here. I'd vote for renaming the 32 bit version to match... I was just going to say the same thing myself. Please do the honors if you have time... -- Peter Geoghegan

Re: GlobalVisIsRemovableFullXid() vs GlobalVisCheckRemovableXid()

2021-02-14 Thread Peter Geoghegan
On Sun, Feb 14, 2021 at 4:21 PM Thomas Munro wrote: > Done. Thanks. -- Peter Geoghegan

Snapshot scalability patch issue

2021-02-14 Thread Peter Geoghegan
s here. I also notice some inconsistencies in the heap_page_prune() prototype names vs the corresponding definition names. Might be worth doing something about in passing. -- Peter Geoghegan

Re: 64-bit XIDs in deleted nbtree pages

2021-02-14 Thread Peter Geoghegan
On Sat, Feb 13, 2021 at 10:47 PM Peter Geoghegan wrote: > It will be rare. But more importantly, the fact that scenario is now > an extreme case justifies treating it as an extreme case. We can teach > _bt_vacuum_needs_cleanup() to recognize it as an extreme case, too. In > particu

Re: snowball update

2021-02-15 Thread Peter Eisentraut
On 2021-02-12 22:00, Oleg Bartunov wrote: We don't have (and really it's impossible) regression test for stemmers, so maybe we should warn users about possible inconsistencies of old tsvectors and new stemmers ? Yeah, it's analogous to collation and Unicode updates. We could invent a versioni

SSL SNI

2021-02-15 Thread Peter Eisentraut
5 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Mon, 15 Feb 2021 14:11:27 +0100 Subject: [PATCH] Set SNI for SSL connections from the client This allows an SNI-aware proxy to route connections. --- src/interfaces/libpq/fe-secure-openssl.c | 19 +++ 1 file changed, 19 inse

Re: Improve new hash partition bound check error messages

2021-02-15 Thread Peter Eisentraut
On 2021-02-03 15:52, Peter Eisentraut wrote: On 2021-02-02 13:26, Heikki Linnakangas wrote: How about this? CREATE TABLE fail_part PARTITION OF hash_parted FOR VALUES WITH (MODULUS 25, REMAINDER 3); ERROR: every hash partition modulus must be a factor of the next larger modulus DETAIL: 25 is

Re: SSL SNI

2021-02-15 Thread Peter Eisentraut
On 2021-02-15 18:40, Jesse Zhang wrote: I imagine this also (finally) opens up the possibility for the server to present a different certificate for each hostname based on SNI. This eliminates the requirement for wildcard certs where the cluster is running on a host with multiple (typically two t

Re: [HACKERS] logical decoding of two-phase transactions

2021-02-15 Thread Peter Smith
for the "two_phase" option. So: a) should I add a new test per your feedback comment, or b) should I be consistent with the other similar errors, and not add the test? Of course it is easy to add a new test if you think option (a) is best. Thoughts? - Kind Regards, Peter Smith. Fujitsu Australia

Re: Snapshot scalability patch issue

2021-02-15 Thread Peter Geoghegan
On Mon, Feb 15, 2021 at 5:30 PM Andres Freund wrote: > Done. Thanks for noticing/reporting! Great, thanks! -- Peter Geoghegan

Re: 64-bit XIDs in deleted nbtree pages

2021-02-15 Thread Peter Geoghegan
e units of free space (heapam is another story). A 100% crash-safe design would naturally shift the problem of nbtree page recycle safety from the producer/VACUUM side, to the consumer/_bt_getbuf() side, which I agree would be a real improvement. But these long standing FSM issues are not going to change for Postgres 14. And so changing _bt_getbuf() to do clever things with XIDs won't be possible for Postgres 14 IMV. -- Peter Geoghegan

Re: 64-bit XIDs in deleted nbtree pages

2021-02-15 Thread Peter Geoghegan
On Mon, Feb 15, 2021 at 7:26 PM Peter Geoghegan wrote: > Actually, there is one more reason why I bring up idea 1 now: I want > to hear your thoughts on the index AM API questions now, which idea 1 > touches on. Ideally all of the details around the index AM VACUUM APIs > (i.e. when a

Re: 64-bit XIDs in deleted nbtree pages

2021-02-16 Thread Peter Geoghegan
described? I assumed that that was okay in general, but I haven't tested parallel VACUUM specifically. Will parallel VACUUM really fail to ensure that values in bulk stats fields (like pages_deleted and pages_free) get set correctly for amvacuumcleanup() callbacks? -- Peter Geoghegan

Re: 64-bit XIDs in deleted nbtree pages

2021-02-16 Thread Peter Geoghegan
On Tue, Feb 16, 2021 at 11:35 AM Peter Geoghegan wrote: > Isn't btvacuumcleanup() (or any other amvacuumcleanup() routine) > entitled to rely on the bulk delete stats being set in the way I've > described? I assumed that that was okay in general, but I haven't

Finding cause of test fails on the cfbot site

2021-02-17 Thread Peter Smith
://cfbot.cputube.org/ [2] https://api.cirrus-ci.com/v1/task/5352561114873856/logs/test.log Kind Regards, Peter Smith. Fujitsu Australia

Re: Finding cause of test fails on the cfbot site

2021-02-17 Thread Peter Smith
st#L733 Thanks for all the effort spent into looking at this. Meanwhile, since you pointed out the patch is not applying on the HEAD tip I can at least address that. -- Kind Regards, Peter Smith. Fujitsu Australia

Re: Is it worth accepting multiple CRLs?

2021-02-17 Thread Peter Eisentraut
On 2021-02-17 05:05, Kyotaro Horiguchi wrote: The commit fe61df7f82 shot down this. This patch allows a new GUC ssl_crl_dir and a new libpq connection option sslcrldir to specify CRL directory, which stores multiple files that contains one CRL. With that method server loads only CRLs for the CA

Re: Printing LSN made easy

2021-02-18 Thread Peter Eisentraut
Here is an updated patch that just introduces LSN_FORMAT_ARGS(). I think the result is quite pleasant. -- Peter Eisentraut 2ndQuadrant, an EDB company https://www.2ndquadrant.com/ From 91d831518c8a71b010bdc835caacaedb20ffb896 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Thu, 18 Feb

cursor sensitivity misunderstanding

2021-02-18 Thread Peter Eisentraut
I think our documentation is mistaken about what it means for a cursor to be "sensitive" or "insensitive". The definition in SQL:2016 is: A change to SQL-data is said to be independent of a cursor CR if and only if it is not made by an or a that is positioned on CR. A change

Re: cursor sensitivity misunderstanding

2021-02-18 Thread Peter Eisentraut
On 18.02.21 17:11, David G. Johnston wrote: The OP was doing a course based on Oracle and was confused regarding our behavior.  The documentation failed to help me provide a useful response, so I'd agree there is something here that needs reworking if not outright fixing. According to the p

Re: Finding cause of test fails on the cfbot site

2021-02-19 Thread Peter Smith
ks! -- [1] http://cfbot.cputube.org/ [2] https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest/32/2914 Kind Regards, Peter Smith. Fujitsu Australia

Re: Improve new hash partition bound check error messages

2021-02-21 Thread Peter Eisentraut
On 15.02.21 17:45, Peter Eisentraut wrote: On 2021-02-03 15:52, Peter Eisentraut wrote: On 2021-02-02 13:26, Heikki Linnakangas wrote: How about this? CREATE TABLE fail_part PARTITION OF hash_parted FOR VALUES WITH (MODULUS 25, REMAINDER 3); ERROR:  every hash partition modulus must be a

Re: SEARCH and CYCLE clauses

2021-02-22 Thread Peter Eisentraut
On 22.05.20 14:32, Peter Eisentraut wrote: As an improvement over the spec, I think the vast majority of people will be using simple true/false values.  Can we make that optional? CYCLE f, t SET is_cycle USING path would be the same as CYCLE f, t SET is_cycle TO true DEFAULT false

Re: SEARCH and CYCLE clauses

2021-02-22 Thread Peter Eisentraut
On 22.02.21 11:05, Vik Fearing wrote: This looks good to me, except that you forgot to add the feature stamp. Attached is a small diff to apply on top of your patch to fix that. The feature code is from SQL:202x, whereas the table is relative to SQL:2016. We could add it, but probably with

Re: 64-bit XIDs in deleted nbtree pages

2021-02-22 Thread Peter Geoghegan
no updates or deletes since the last VACUUM -- that seems like an existing problem worth fixing now. It's way too unclear that this setting only really concerns append-only tables. -- Peter Geoghegan

<    1   2   3   4   5   6   7   8   9   10   >