Re: GiST index build missing smgrimmedsync()?

2022-02-24 Thread Heikki Linnakangas
On 23/02/2022 23:30, Melanie Plageman wrote: I brought this up in [1] but wanted to start a dedicated thread. Since 16fa9b2b30a357 GiST indexes are not built in shared buffers. However, smgrimmedsync() is not done anywhere and skipFsync=true is always passed to smgrwrite() and smgrextend(). So,

Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths

2022-03-11 Thread Heikki Linnakangas
On 11/03/2022 17:42, Matthias van de Meent wrote: Hi, Xlogreader limits the size of what it considers valid xlog records to MaxAllocSize; but this is not currently enforced in the XLogRecAssemble API. This means it is possible to assemble a record that postgresql cannot replay. Oops, that woul

Re: Deleting older versions in unique indexes to avoid page splits

2020-12-01 Thread Heikki Linnakangas
This is a wholly new concept with a lot of heuristics. It's a lot of swallow. But here are a few quick comments after a first read-through: On 30/11/2020 21:50, Peter Geoghegan wrote: +/* + * State used when calling table_index_delete_check() to perform "bottom up" + * deletion of duplicate ind

Re: Avoid using lcons and list_delete_first in plan_union_children()

2020-12-01 Thread Heikki Linnakangas
On 01/12/2020 12:52, Hou, Zhijie wrote: Hi, In function plan_union_children(), I found the lcons and list_delete_first here is easy to be replaced by lappend and list_delete_last. And I also found a previous commit do similar thing, so I try to improve this one. Previous commit: d97b714a2199

Re: Index Skip Scan (new UniqueKeys)

2020-12-01 Thread Heikki Linnakangas
On 01/12/2020 22:21, Dmitry Dolgov wrote: On Mon, Nov 30, 2020 at 04:42:20PM +0200, Heikki Linnakangas wrote: I had a quick look at this patch. I haven't been following this thread, so sorry if I'm repeating old arguments, but here we go: Thanks! - I'm surprised you need

Re: Corner-case bug in pg_rewind

2020-12-02 Thread Heikki Linnakangas
node_2->data_dir; +my $node_1_connstr = $node_1->connstr; + +command_checks_all( + [ +'pg_rewind', +"--source-server=$node_1_connstr", +"--target-pgdata=$node_2_pgdata", +'--dry-run' +], +0, +[], +[qr|rewinding from

Deprecate custom encoding conversions

2020-12-02 Thread Heikki Linnakangas
Hi, PostgreSQL allows writing custom encoding conversion functions between any character encodings, using the CREATE CONVERSION command. It's pretty flexible, you can define default and non-default conversions, and the conversions live in schemas so you can have multiple conversions installed

Re: Deprecate custom encoding conversions

2020-12-02 Thread Heikki Linnakangas
On 02/12/2020 18:18, Tom Lane wrote: Heikki Linnakangas writes: I propose that we add a notice to the CREATE CONVERSION docs to say that it is deprecated, and remove it in a few years. While I agree that it's probably not that useful, what would we gain by removing it? If you inte

Re: Corner-case bug in pg_rewind

2020-12-03 Thread Heikki Linnakangas
On 02/12/2020 15:26, Ian Barwick wrote: On 02/12/2020 20:13, Heikki Linnakangas wrote: Attached are two patches. The first patch is your original patch, unmodified (except for a cosmetic rename of the test file). The second patch builds on that, demonstrating and fixing the issue I mentioned

Re: Corner-case bug in pg_rewind

2020-12-03 Thread Heikki Linnakangas
On 03/12/2020 16:49, Pavel Borisov wrote: Ok, pushed and backpatched this now. Very nice! Thanks to you all! Thanks for the review, Pavel! I just realized that I forgot to credit you in the commit message. I'm sorry. - Heikki

Re: Corner-case bug in pg_rewind

2020-12-03 Thread Heikki Linnakangas
On 03/12/2020 16:10, Heikki Linnakangas wrote: On 02/12/2020 15:26, Ian Barwick wrote: On 02/12/2020 20:13, Heikki Linnakangas wrote: Attached are two patches. The first patch is your original patch, unmodified (except for a cosmetic rename of the test file). The second patch builds on that

Re: Corner-case bug in pg_rewind

2020-12-04 Thread Heikki Linnakangas
On 04/12/2020 00:16, Heikki Linnakangas wrote: On 03/12/2020 16:10, Heikki Linnakangas wrote: On 02/12/2020 15:26, Ian Barwick wrote: On 02/12/2020 20:13, Heikki Linnakangas wrote: Attached are two patches. The first patch is your original patch, unmodified (except for a cosmetic rename of

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-12-05 Thread Heikki Linnakangas
On 05/12/2020 17:10, Andy Fan wrote: Actually I can't understand this, could you explain more?  Based on my current knowledge,  when we run "SELECT DISTINCT a FROM t",  we never care about which operator to use for the unique. SortGroupClause includes 'eqop' field, which determines the opera

pg_rewind race condition just after promotion

2020-12-07 Thread Heikki Linnakangas
573dd387a03e36eda15cb41658447 Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Mon, 7 Dec 2020 19:59:50 +0200 Subject: [PATCH 1/1] pg_rewind: Fix determining TLI when server was just promoted. If the source server was just promoted, and it hasn't written the checkpoint record yet,

Re: pg_rewind race condition just after promotion

2020-12-09 Thread Heikki Linnakangas
On 08/12/2020 06:45, Kyotaro Horiguchi wrote: At Mon, 7 Dec 2020 20:13:25 +0200, Heikki Linnakangas wrote in I think we should fix this properly. I'm not sure if it can lead to a broken cluster, but at least it can cause pg_rewind to fail unnecessarily and in a user-unfriendly way. But th

Re: Feature Proposal: Add ssltermination parameter for SNI-based LoadBalancing

2020-12-11 Thread Heikki Linnakangas
On 10/12/2020 17:49, Lukas Meisegeier wrote: I try to host multiple postgresql-servers on the same ip and the same port through SNI-based load-balancing. Currently this is not possible because of two issues: 1. The psql client won't set the tls-sni-extension correctly (https://www.postgresql.org/

Re: Feature Proposal: Add ssltermination parameter for SNI-based LoadBalancing

2020-12-11 Thread Heikki Linnakangas
On 11/12/2020 16:46, Lukas Meisegeier wrote: Hey Heikki, thanks for providing feedback :) The traffic between proxy and psql-server is unencrypted thats why I don't need to patch the server. Ok. I tried returning a fixed response on the first plain SSLRequest forwarding it to a psql-server w

Re: pg_basebackup caused FailedAssertion

2020-12-14 Thread Heikki Linnakangas
On 12/12/2020 00:47, Jeff Davis wrote: On Wed, 2013-02-27 at 19:29 +0200, Heikki Linnakangas wrote: Right. I fixed that by adding WL_SOCKET_READABLE, and handling any messages that might arrive after the frontend already sent CopyEnd. The frontend shouldn't send any messages after Co

Re: anonymous block returning like a function

2020-12-14 Thread Heikki Linnakangas
On 11/12/2020 21:06, PegoraroF10 wrote: I would like to have an anonymous block, like DO, but having resuts, like an usual function does. I know any user can do ... create function pg_temp.run_time_bigger(numeric,numeric) returns numeric language plpgsql as $$ begin if $1 > $2 then return $1; e

Re: Feature Proposal: Add ssltermination parameter for SNI-based LoadBalancing

2020-12-14 Thread Heikki Linnakangas
On 12/12/2020 13:52, Lukas Meisegeier wrote: Thanks for the provided ideas :) I use HaProxy for my load-balancing and unfortunately I can't define that I want to listen on a port for both ssl and non ssl requests. Could you configure HaProxy to listen on separate ports for SSL and non-SSL conn

Re: Optimizing the documentation

2020-12-14 Thread Heikki Linnakangas
On 14/12/2020 21:50, Joshua Drake wrote: The community has spent a lot of time optimizing features over the years. Excellent examples include parallel query and partitioning which have been multi-year efforts to increase the quality,  performance, and extend features of the original commit. We

Re: ResourceOwner refactoring

2020-12-16 Thread Heikki Linnakangas
ourceOwnerRememberBBB() will find an empty slot. I don't think we do that currently, but to be sure, I'm planning to grep ResourceOwnerRemember and look at each call carefullly. And perhaps we can add an assertion for this, although I'm not sure where. - Heikki >From 71a372b

Re: pgbench failed when -f option contains a char '@'

2020-12-18 Thread Heikki Linnakangas
On 18/12/2020 08:22, Wang, Shenhao wrote: Hi, hackers pgbench use -f filename[@weight] to receive a sql script file with a weight, but if I create a file contains char'@', like a...@2.sql, specify this file without weigth, pgbench will failed with error: pgbench: fatal: invalid weight s

Re: Incorrect allocation handling for cryptohash functions with OpenSSL

2020-12-18 Thread Heikki Linnakangas
On 18/12/2020 09:35, Michael Paquier wrote: Hi all, As of the work done in 87ae9691, I have played with error injections in the code paths using this code, but forgot to count for cases where cascading resowner cleanups are involved. Like other resources (JIT, DSM, etc.), this requires an alloc

Re: Incorrect allocation handling for cryptohash functions with OpenSSL

2020-12-18 Thread Heikki Linnakangas
On 18/12/2020 11:35, Heikki Linnakangas wrote: BTW, looking at pg_cryptohash_ctx and pg_cryptohash_state, why do we need two structs? They're both allocated and controlled by the cryptohash implementation. It would seem simpler to have just one. Something like this. Passes regression

Re: Refactor routine to check for ASCII-only case

2020-12-18 Thread Heikki Linnakangas
On 18/12/2020 05:57, Michael Paquier wrote: As mentioned in [1], there are three places where there is the same routine to check if a string is made only of ASCII characters. This makes for a small-ish but nice cleanup, as per the attached. +1 - Heikki

Re: Incorrect allocation handling for cryptohash functions with OpenSSL

2020-12-18 Thread Heikki Linnakangas
On 18/12/2020 12:10, Michael Paquier wrote: On Fri, Dec 18, 2020 at 11:35:14AM +0200, Heikki Linnakangas wrote: pg_cryptohash_create() is now susceptible to leaking memory in TopMemoryContext, if the allocations fail. I think the attached should fix it (but I haven't tested it at all).

Re: Incorrect allocation handling for cryptohash functions with OpenSSL

2020-12-18 Thread Heikki Linnakangas
On 18/12/2020 12:55, Heikki Linnakangas wrote: On 18/12/2020 12:10, Michael Paquier wrote: On Fri, Dec 18, 2020 at 11:35:14AM +0200, Heikki Linnakangas wrote: pg_cryptohash_create() is now susceptible to leaking memory in TopMemoryContext, if the allocations fail. I think the attached should

Re: \gsetenv

2020-12-20 Thread Heikki Linnakangas
On 20/12/2020 21:05, David Fetter wrote: We have plenty of ways to spawn shells and cause havoc, and we wouldn't be able to block them all even if we decided to put a bunch of pretty onerous restrictions on psql at this very late date. We have \set, backticks, \!, and bunches of things less obvio

Re: Perform COPY FROM encoding conversions in larger chunks

2020-12-22 Thread Heikki Linnakangas
On 22/12/2020 22:01, John Naylor wrote: In 0004, it seems you have some doubts about upgrade compatibility. Is that because user-defined conversions would no longer have the right signature? Exactly. If you have an extension that adds a custom conversion function and does CREATE CONVERSION, t

Re: Deleting older versions in unique indexes to avoid page splits

2021-01-04 Thread Heikki Linnakangas
On 02/12/2020 00:18, Peter Geoghegan wrote: On Tue, Dec 1, 2020 at 1:50 AM Heikki Linnakangas wrote: On 30/11/2020 21:50, Peter Geoghegan wrote: +} TM_IndexDelete; +} TM_IndexStatus; Is it really significantly faster to have two arrays? If you had just one array, each element would be

Re: pg_rewind restore_command issue in PG12

2021-01-04 Thread Heikki Linnakangas
On 03/01/2021 20:13, Amine Tengilimoglu wrote:      In a situation where pg_rewind gets an error due to a missing wall, I  have set restore_command so that the needed wals can be read from the archive (I don't want to manually copy the wal files), but I see it doesn't work. What am I missing? 

Re: Incorrect allocation handling for cryptohash functions with OpenSSL

2021-01-06 Thread Heikki Linnakangas
On 06/01/2021 13:42, Michael Paquier wrote: On Sat, Dec 19, 2020 at 03:13:50PM +0900, Michael Paquier wrote: At the same time, I have taken care of your comment from upthread to return a failure if the caller passes NULL for the context, and adjusted some comments. What do you think of the atta

Re: Incorrect allocation handling for cryptohash functions with OpenSSL

2021-01-06 Thread Heikki Linnakangas
On 25/12/2020 02:57, Michael Paquier wrote: On Mon, Dec 21, 2020 at 04:28:26PM -0500, Robert Haas wrote: TBH, I think there's no point in return an error here at all, because it's totally non-specific. You have no idea what failed, just that something failed. Blech. If we want to check that ctx

Re: Incorrect allocation handling for cryptohash functions with OpenSSL

2021-01-06 Thread Heikki Linnakangas
On 07/01/2021 06:15, Michael Paquier wrote: On Wed, Jan 06, 2021 at 03:58:22PM +0200, Heikki Linnakangas wrote: contrib/pgcrypto/internal-sha2.c and src/backend/utils/adt/cryptohashfuncs.c: the call to pg_cryptohash_create() is missing check for NULL result. With your latest patch, that'

Re: [POC] verifying UTF-8 using SIMD instructions

2021-02-15 Thread Heikki Linnakangas
On 13/02/2021 03:31, John Naylor wrote: On Mon, Feb 8, 2021 at 6:17 AM Heikki Linnakangas <mailto:hlinn...@iki.fi>> wrote: > > I also tested the fallback implementation from the simdjson library > (included in the patch, if you uncomment it in simdjson-glue.c): &g

Re: Extensibility of the PostgreSQL wire protocol

2021-02-19 Thread Heikki Linnakangas
On 19/02/2021 14:29, Damir Simunic wrote: On 11 Feb 2021, at 16:06, Tom Lane wrote: Maybe there is some useful thing that can be accomplished here, but we need to consider the bigger picture rather than believing (without proof) that a few hook variables will be enough to do anything. Plugg

Re: Remove latch.c workaround for Linux < 2.6.27

2021-02-27 Thread Heikki Linnakangas
On 27 February 2021 01:10:23 EET, Thomas Munro wrote: >Hello, > >Commit 82ebbeb0 added a workaround for (already in 2017) ancient Linux >kernels with no EPOLL_CLOEXEC. I don't see any such systems in the >build farm today (and if there is one hiding in there somewhere, it's >well past time to upg

Re: PITR promote bug: Checkpointer writes to older timeline

2021-03-03 Thread Heikki Linnakangas
On 03/03/2021 08:47, Kyotaro Horiguchi wrote: At Tue, 2 Mar 2021 17:56:03 -0800, Soumyadeep Chakraborty wrote in When there are prepared transactions in an older timeline, in the checkpointer, a call to CheckPointTwoPhase() and subsequently to XlogReadTwoPhaseData() and subsequently to read_lo

Re: Removing support for COPY FROM STDIN in protocol version 2

2021-03-04 Thread Heikki Linnakangas
On 04/03/2021 01:32, Tom Lane wrote: Patched psql, trying to connect to a 7.3 server, reports this: $ psql -h ... psql: error: connection to server at "sss2" (192.168.1.3), port 5432 failed: FATAL: unsupported frontend protocol $ Conversely, 7.3 psql trying to connect to a patched server rep

Force lookahead in COPY FROM parsing

2021-03-04 Thread Heikki Linnakangas
at. - Heikki >From 82680ff836fd5b43d1f83ed01e490d831ffa2b09 Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Thu, 4 Mar 2021 11:10:39 +0200 Subject: [PATCH v2 1/1] Simplify COPY FROM parsing by forcing lookahead. Now that we don't support the old-style COPY protocol anymore, we can force four

Re: Removing support for COPY FROM STDIN in protocol version 2

2021-03-04 Thread Heikki Linnakangas
On 04/03/2021 22:04, Tom Lane wrote: Heikki Linnakangas writes: I concur that 0001 attached is committable. I have not looked at your 0002, though. Removed the entry from nls.mk, and pushed 0001. Thanks! It seems that buildfarm member walleye doesn't like this. Since nothing el

Re: ResourceOwner refactoring

2021-03-09 Thread Heikki Linnakangas
saving rejects to file src/backend/utils/cache/plancache.c.rej 2 out of 15 hunks FAILED -- saving rejects to file src/backend/utils/resowner/resowner.c.rej Can we get a rebase? Here you go. - Heikki >From 8b852723212c51d3329267fcbf8e7a28f49dbbdd Mon Sep 17 00:00:00 2001 From: Heikki Linnakan

Re: Question about make coverage-html

2020-10-27 Thread Heikki Linnakangas
On 27/10/2020 10:09, Peter Smith wrote: Hi hackers. The example of test coverage in the documentation [1] works as advertised. But I wanted to generate test coverage results only of some TAP tests in src/test/subscription. The documentation [1] also says "The make commands also work in subdire

Re: Yet another fast GiST build

2020-10-27 Thread Heikki Linnakangas
On 21/10/2020 20:13, Andrey Borodin wrote: 7 окт. 2020 г., в 17:38, Heikki Linnakangas написал(а): On 07/10/2020 15:27, Andrey Borodin wrote: Here's draft patch with implementation of sortsupport for ints and floats. +static int +gbt_int4_cmp(Datum a, Datum b, SortSupport

Re: partition routing layering in nodeModifyTable.c

2020-10-27 Thread Heikki Linnakangas
On 23/10/2020 12:37, Amit Langote wrote: To explain these numbers a bit, "overheaul update/delete processing" patch improves the performance of that benchmark by allowing the updates to use run-time pruning when executing generic plans, which they can't today. However without "lazy-ResultRelInfo

Re: Support for NSS as a libpq TLS backend

2020-10-27 Thread Heikki Linnakangas
On 27/10/2020 22:07, Daniel Gustafsson wrote: /* * Track whether the NSS database has a password set or not. There is no API * function for retrieving password status, so we simply flip this to true in * case NSS invoked the password callback - as that will only happen in case * there is a pa

Deduplicate aggregates and transition functions in planner

2020-10-28 Thread Heikki Linnakangas
27;m not sure yet if we should accept the new plan, or change the query to still get the old plan. - Heikki >From 0100f5e477af814afa23bf7933393f4b144b835f Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Wed, 28 Oct 2020 20:58:14 +0200 Subject: [PATCH v1 1/1] Move peragg and pertran

Re: Deduplicate aggregates and transition functions in planner

2020-10-29 Thread Heikki Linnakangas
On 28/10/2020 21:59, Andres Freund wrote: On 2020-10-28 21:10:41 +0200, Heikki Linnakangas wrote: Currently, ExecInitAgg() performs quite a lot of work, to deduplicate identical Aggrefs, as well as Aggrefs that can share the same transition state. That doesn't really belong in the executo

Re: Parallel copy

2020-10-29 Thread Heikki Linnakangas
On 27/10/2020 15:36, vignesh C wrote: Attached v9 patches have the fixes for the above comments. I did some testing: /tmp/longdata.pl: #!/usr/bin/perl # # Generate three rows: # foo # longdatalongdatalongdata... # bar # # The length of the middle row is given as command line arg. # m

Re: Parallel copy

2020-10-30 Thread Heikki Linnakangas
On 27/10/2020 15:36, vignesh C wrote: Attached v9 patches have the fixes for the above comments. I find this design to be very complicated. Why does the line-boundary information need to be in shared memory? I think this would be much simpler if each worker grabbed a fixed-size block of raw d

Re: Parallel copy

2020-10-30 Thread Heikki Linnakangas
On 30/10/2020 18:36, Heikki Linnakangas wrote: I find this design to be very complicated. Why does the line-boundary information need to be in shared memory? I think this would be much simpler if each worker grabbed a fixed-size block of raw data, and processed that. In your patch, the leader

Re: Parallel copy

2020-10-30 Thread Heikki Linnakangas
On 30/10/2020 18:36, Heikki Linnakangas wrote: Whether the leader process finds the EOLs or the worker processes, it's pretty clear that it needs to be done ASAP, for a chunk at a time, because that cannot be done in parallel. I think some refactoring in CopyReadLine() and friends would

Re: making update/delete of inheritance trees scale better

2020-10-30 Thread Heikki Linnakangas
On 29/10/2020 15:03, Amit Langote wrote: On Sun, Oct 4, 2020 at 11:44 AM Amit Langote wrote: On Fri, Sep 11, 2020 at 7:20 PM Amit Langote wrote: Here are the commit messages of the attached patches: [PATCH v3 1/3] Overhaul how updates compute a new tuple I tried to assess the performance i

Re: making update/delete of inheritance trees scale better

2020-10-30 Thread Heikki Linnakangas
On 30/10/2020 23:10, Tom Lane wrote: Heikki Linnakangas writes: I also did some quick performance testing with a simple update designed as a worst-case scenario: vacuum tab; update tab set b = b, a = a; In this case, the patch fetches the old tuple, but it wouldn't really ne

Re: Parallel copy

2020-10-30 Thread Heikki Linnakangas
On 30/10/2020 22:56, Tomas Vondra wrote: I agree this design looks simpler. I'm a bit worried about serializing the parsing like this, though. It's true the current approach (where the first phase of parsing happens in the leader) has a similar issue, but I think it would be easier to improve tha

Re: making update/delete of inheritance trees scale better

2020-10-30 Thread Heikki Linnakangas
On 31/10/2020 00:12, Tom Lane wrote: Heikki Linnakangas writes: But if you do: postgres=# explain verbose update tab set a = 1, b = 2; QUERY PLAN - Update on

Re: Parallel copy

2020-11-01 Thread Heikki Linnakangas
On 02/11/2020 08:14, Amit Kapila wrote: On Fri, Oct 30, 2020 at 10:11 PM Heikki Linnakangas wrote: Leader process: The leader process is simple. It picks the next FREE buffer, fills it with raw data from the file, and marks it as FILLED. If no buffers are FREE, wait. Worker process: 1

Re: Parallel copy

2020-11-01 Thread Heikki Linnakangas
On 02/11/2020 09:10, Heikki Linnakangas wrote: On 02/11/2020 08:14, Amit Kapila wrote: We have discussed both these approaches (a) single producer multiple consumer, and (b) all workers doing the processing as you are saying in the beginning and concluded that (a) is better, see some of the

Re: Fix a typo in verify_heapam.c

2020-11-02 Thread Heikki Linnakangas
On 02/11/2020 10:05, Masahiko Sawada wrote: Hi all, I've attached a patch that fixes a typo in verify_heapam.c recently added. s/wheter/whether/ This was just fixed in commit 8a15e735b. - Heikki

Re: [patch] [doc] Add SELECT clause literals to queries section headers

2020-11-02 Thread Heikki Linnakangas
On 21/10/2020 18:04, David G. Johnston wrote: Hackers, Re-sending from -docs [1] with attachment in order to add to commitfest. David J. [1] https://www.postgresql.org/message-id/flat/159981394174.31338.7014519396749859167%40wrigleys.postgresql.org edit: attaching the patch Seems like a go

Re: Getting rid of aggregate_dummy()

2020-11-02 Thread Heikki Linnakangas
On 01/11/2020 22:47, Tom Lane wrote: With that, we don't actually need aggregate_dummy() to exist at all, because it's never referenced. Having "aggregate_dummy" as the prosrc value for an aggregate function is now just a random convention; any other string would do as well. (We could save a fe

Re: hash_array_extended() needs to pass down collation

2020-11-02 Thread Heikki Linnakangas
On 02/11/2020 09:40, Peter Eisentraut wrote: I noticed that hash_array_extended() does not pass down the collation to the element's collation function, unlike hash_array(). As a consequence, hash partitioning using text arrays as partition key fails. The attached patch fixes this. I propose to

Re: ModifyTable overheads in generic plans

2020-11-02 Thread Heikki Linnakangas
On 30/10/2020 08:13, Amit Langote wrote: /* * Perform WITH CHECK OPTIONS check, if any. */ static void ExecProcessWithCheckOptions(ModifyTableState *mtstate, ResultRelInfo *resultRelInfo, TupleTableSlot *slot, WCOKind wco_kind) {

Re: Yet another fast GiST build

2020-11-02 Thread Heikki Linnakangas
On 30/10/2020 20:20, Andrey Borodin wrote: 27 окт. 2020 г., в 16:43, Heikki Linnakangas написал(а): static bool gbt_inet_abbrev_abort(int memtupcount, SortSupport ssup) { #if SIZEOF_DATUM == 8 return false; #else return true; #endif } Better to not set the 'abbrev_conv

Re: Split copy.c

2020-11-02 Thread Heikki Linnakangas
On 02/11/2020 19:23, Andres Freund wrote: On 2020-11-02 11:03:29 +0200, Heikki Linnakangas wrote: There isn't much common code between COPY FROM and COPY TO, so I propose that we split copy.c into two: copyfrom.c and copyto.c. See attached. I thin that's much nicer. Not quite convin

Re: ModifyTable overheads in generic plans

2020-11-03 Thread Heikki Linnakangas
On 03/11/2020 10:27, Amit Langote wrote: Please check the attached if that looks better. Great, thanks! Yeah, I like that much better. This makes me a bit unhappy: /* Also let FDWs init themselves for foreign-table result rels */ if (resultRelInfo->ri_FdwRou

Re: Parallel copy

2020-11-03 Thread Heikki Linnakangas
On 03/11/2020 10:59, Amit Kapila wrote: On Mon, Nov 2, 2020 at 12:40 PM Heikki Linnakangas wrote: However, the point of parallel copy is to maximize bandwidth. Okay, but this first-phase (finding the line boundaries) can anyway be not done in parallel and we have seen in some of the initial

Re: [PATCH] remove pg_archivecleanup and pg_standby

2020-11-03 Thread Heikki Linnakangas
On 02/11/2020 20:26, Justin Pryzby wrote: On Thu, Oct 29, 2020 at 08:40:31PM +0100, Michael Banck wrote: Am Mittwoch, den 28.10.2020, 21:44 -0500 schrieb Justin Pryzby: Forking this thread: https://www.postgresql.org/message-id/fd93f1c5-7818-a02c-01e5-1075ac0d4...@iki.fi I think these are ol

Re: Refactor pg_rewind code and make it work against a standby

2020-11-04 Thread Heikki Linnakangas
On 25/09/2020 02:56, Soumyadeep Chakraborty wrote: On Thu, Sep 24, 2020 at 10:27 AM Heikki Linnakangas wrote: 7. Please address the FIXME for the symlink case: /* FIXME: Check if it points to the same target? */ It's not a new issue. Would be nice to fix, of course. I'm not sur

Re: Protect syscache from bloating with negative cache entries

2020-11-05 Thread Heikki Linnakangas
On 19/11/2019 12:48, Kyotaro Horiguchi wrote: 1. Inserting a branch in SearchCatCacheInternal. (CatCache_Pattern_1.patch) This is the most straightforward way to add an alternative feature. pattern 1 | 8459.73 | 28.15 # 9% (>> 1%) slower than 7757.58 pattern 1 | 8504.83 | 55.61 pattern 1 |

Re: Protect syscache from bloating with negative cache entries

2020-11-06 Thread Heikki Linnakangas
move other code in memory so that a some hot codepath or struct gets split across CPU cache lines. It can be infuriating when benchmarking. At Thu, 5 Nov 2020 11:09:09 +0200, Heikki Linnakangas wrote in (A) original test patch I naively thought that the code path is too short to bury the degr

Re: ModifyTable overheads in generic plans

2020-11-10 Thread Heikki Linnakangas
On 10/11/2020 13:12, Amit Langote wrote: On Wed, Nov 4, 2020 at 11:32 AM Amit Langote wrote: On Tue, Nov 3, 2020 at 9:05 PM Heikki Linnakangas wrote: A) We could change FDW API so that BeginDirectModify takes the same arguments as BeginForeignModify(). That avoids the assumption that it

Re: ModifyTable overheads in generic plans

2020-11-11 Thread Heikki Linnakangas
On 10/11/2020 17:32, Heikki Linnakangas wrote: On 10/11/2020 13:12, Amit Langote wrote: On second thought, it seems A would amount to merely a cosmetic adjustment of the API, nothing more. B seems to get the job done for me and also doesn't unnecessarily break compatibility, so I'

Re: Clean up optional rules in grammar

2020-11-11 Thread Heikki Linnakangas
On 11/11/2020 11:12, Peter Eisentraut wrote: The attached patch cleans this up to make them all look like the first style. +1 - Heikki

Re: making update/delete of inheritance trees scale better

2020-11-11 Thread Heikki Linnakangas
On 29/10/2020 15:03, Amit Langote wrote: Rebased over the recent executor result relation related commits. ModifyTablePath didn't get the memo that a ModifyTable can only have one subpath after these patches. Attached patch, on top of your v5 patches, cleans that up. - Heikki diff --git a/s

Re: ModifyTable overheads in generic plans

2020-11-11 Thread Heikki Linnakangas
I'm still a bit confused and unhappy about the initialization of ResultRelInfos and the various fields in them. We've made progress in the previous patches, but it's still a bit messy. /* * If transition tuples will be captured, initialize a map to convert

Re: Refactor pg_rewind code and make it work against a standby

2020-11-12 Thread Heikki Linnakangas
On 04/11/2020 11:23, Heikki Linnakangas wrote: I read through the patches one more time, fixed a bunch of typos and such, and pushed patches 1-4. I'm going to spend some more time on testing the last patch. It allows using a standby server as the source, and we don't have any tests fo

pg_rewind copies

2020-11-13 Thread Heikki Linnakangas
one half from the new. That's a much more narrow window, though, and pg_basebackup has the same problem. - Heikki >From f0207e355c6be2d710d2ac11e4895795c3471ef3 Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Fri, 13 Nov 2020 11:04:29 +0200 Subject: [PATCH 1/1] pg_rewind: Fetch sma

Re: [PATCH] Combine same ternary types in GIN and TSearch

2020-11-13 Thread Heikki Linnakangas
On 13/11/2020 11:04, Pavel Borisov wrote: Hi, hackers! For historical reasons, now we have two differently named but similar ternary data types in TSearch and Gin text-related types. Before v13 there was also Gin's private TS_execute() version, from which we eventually shifted to Tsearch's TS

Re: Online checksums patch - once again

2020-11-13 Thread Heikki Linnakangas
On 12/11/2020 15:17, Daniel Gustafsson wrote: On 5 Oct 2020, at 14:14, Heikki Linnakangas wrote: I would expect the checksums worker to be automatically started at postmaster startup. Can we make that happen? A dynamic background worker has to be registered from a regular backend, so it&#

Re: Refactor pg_rewind code and make it work against a standby

2020-11-15 Thread Heikki Linnakangas
On 15/11/2020 09:07, Tom Lane wrote: I wrote: Not sure if you noticed, but piculet has twice failed the 007_standby_source.pl test that was added by 9c4f5192f: ... Now, I'm not sure what to make of that, but I can't help noticing that piculet uses --disable-atomics while francolin uses --disable

Re: Split copy.c

2020-11-16 Thread Heikki Linnakangas
On 16/11/2020 04:28, Justin Pryzby wrote: On Tue, Nov 03, 2020 at 03:15:27PM +1300, David Rowley wrote: On Tue, 3 Nov 2020 at 07:35, Andres Freund wrote: On 2020-11-02 19:43:38 +0200, Heikki Linnakangas wrote: On 02/11/2020 19:23, Andres Freund wrote: On 2020-11-02 11:03:29 +0200, Heikki

Re: [patch] [doc] Clarify that signal functions have no feedback

2020-11-17 Thread Heikki Linnakangas
On 02/11/2020 18:02, David G. Johnston wrote: diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index bf6004f321..43bc2cf086 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -23892,7 +23892,8 @@ SELECT collation for ('foo' COLLATE "de_DE"); The functions

ResourceOwner refactoring

2020-11-17 Thread Heikki Linnakangas
Two recent patches that I have reviewed have reminded me that the ResourceOwner interface is a bit clunky. There are two issues: 1. Performance. It's fast enough in most use, but when I was testing Kyotaro's catcache patches [1], the Resowner functions to track catcache references nevertheless

Re: Protect syscache from bloating with negative cache entries

2020-11-17 Thread Heikki Linnakangas
On 09/11/2020 11:34, Kyotaro Horiguchi wrote: At Fri, 6 Nov 2020 10:42:15 +0200, Heikki Linnakangas wrote in Do you need the "ntaccess == 2" test? You could always increment the counter, and in the code that uses ntaccess to decide what to evict, treat all values >= 2 the same. N

Re: [patch] [doc] Clarify that signal functions have no feedback

2020-11-18 Thread Heikki Linnakangas
On 17/11/2020 21:50, David G. Johnston wrote: On Tue, Nov 17, 2020 at 6:13 AM Heikki Linnakangas <mailto:hlinn...@iki.fi>> wrote: You can use the pg_file_settings and pg_hba_file_rules views to check the configuration files for possible errors, before reloading. I agree wi

Re: ResourceOwner refactoring

2020-11-18 Thread Heikki Linnakangas
On 18/11/2020 10:06, Michael Paquier wrote: On Tue, Nov 17, 2020 at 04:21:29PM +0200, Heikki Linnakangas wrote: Attached patch refactors the ResourceOwner internals to do that. + * Size of the small fixed-size array to hold most-recently remembered resources. */ -#define

Re: Add LWLock blocker(s) information

2020-11-18 Thread Heikki Linnakangas
On 11/08/2020 03:41, Andres Freund wrote: On 2020-08-10 18:27:17 -0400, Robert Haas wrote: On Tue, Jun 2, 2020 at 8:25 AM Drouvot, Bertrand wrote: the patch adds into the LWLock struct: last_holding_pid: last pid owner of the lock last_mode: last hold

Re: don't allocate HashAgg hash tables when running explain only

2020-11-18 Thread Heikki Linnakangas
On 13/11/2020 18:10, Alexey Bashtanov wrote: Hi, I got somewhat scared when my explain took a few seconds to complete and used a few gigs of RAM. To reproduce try the following: discard temp; create temp table a as select to_timestamp(generate_series(1, 7000)) i; analyze a; set work_mem to '3GB

Re: Devel docs on website reloading

2020-11-18 Thread Heikki Linnakangas
On 18/11/2020 12:56, Magnus Hagander wrote: Today, we build and update the docs on https://www.postgresql.org/docs/devel/ are rebuilt and deployed by the buildfarm automatically every 4 hours. If there are no changes at all made to the docs, they are *still* kicked out of all caches and the sear

Re: don't allocate HashAgg hash tables when running explain only

2020-11-18 Thread Heikki Linnakangas
On 19/11/2020 07:20, Greg Stark wrote: On Wed, 18 Nov 2020 at 05:40, Heikki Linnakangas wrote: On 13/11/2020 18:10, Alexey Bashtanov wrote: I would appreciate if someone could have a look at the patch attached, which makes executor skip initializing hash tables when doing explain only

Re: Deduplicate aggregates and transition functions in planner

2020-11-19 Thread Heikki Linnakangas
On 29/10/2020 19:48, Andres Freund wrote: On 2020-10-29 10:17:20 +0200, Heikki Linnakangas wrote: On 28/10/2020 21:59, Andres Freund wrote: It wouldn't surprise me to see a small execution time speedup here - I've seen the load of the aggno show up in profiles. I think you'd

Re: don't allocate HashAgg hash tables when running explain only

2020-11-20 Thread Heikki Linnakangas
On 20/11/2020 08:31, Michael Paquier wrote: On Thu, Nov 19, 2020 at 08:47:51AM +0200, Heikki Linnakangas wrote: Yeah, I believe it's always been like that. Yeah, arguably it should be backpatched. I felt conservative and didn't backpatch, but feel free to do it if you think it shoul

Re: Refactor pg_rewind code and make it work against a standby

2020-11-20 Thread Heikki Linnakangas
On 20/11/2020 02:38, Andres Freund wrote: I locally, on a heavily modified branch (AIO support), started to get consistent failures in this test. I *suspect*, but am not sure, that it's the test's fault, not the fault of modifications. As far as I can tell, after the pg_rewind call, there's no g

Re: Refactor pg_rewind code and make it work against a standby

2020-11-20 Thread Heikki Linnakangas
On 20/11/2020 19:14, Andres Freund wrote: Hi, On 2020-11-20 16:19:03 +0200, Heikki Linnakangas wrote: Pushed a fix similar to your patch, but I put the wait_for_catchup() before running pg_rewind. The point of inserting the 'in A, after C was promoted' row is that it's pr

Re: bug in pageinspect's "tuple data" feature

2020-11-22 Thread Heikki Linnakangas
On 21/11/2020 21:32, Alvaro Herrera wrote: If you have a sufficiently broken data page, pageinspect throws an error when trying to examine the page: ERROR: invalid memory alloc request size 18446744073709551451 This is pretty unhelpful; it would be better not to try to print the data instead o

Re: Split copy.c

2020-11-23 Thread Heikki Linnakangas
On 18/11/2020 08:21, Soumyadeep Chakraborty wrote: On Tue, Nov 17, 2020 at 2:38 AM Heikki Linnakangas wrote: Fixed all the other things you listed, fixed a bug in setting 'file_encoding', and trimmed down the #includes. Thanks! LGTM! Marking as Ready for Committer. Pushed, than

Re: Multi Inserts in CREATE TABLE AS - revived patch

2020-11-23 Thread Heikki Linnakangas
On 23/11/2020 11:15, Bharath Rupireddy wrote: Attaching v2 patch, rebased on the latest master 17958972. I just broke this again with commit c532d15ddd to split up copy.c. Here's another rebased version. - Heikki >From dca55175c590914f6a802ec3d36e20db55a3e3c7 Mon Sep 17 00:00:00 2001 From: B

prion failed with ERROR: missing chunk number 0 for toast value 14334 in pg_toast_2619

2020-11-23 Thread Heikki Linnakangas
Hi, After my commit c532d15ddd to split up copy.c, buildfarm animal "prion" failed in pg_upgrade (https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prion&dt=2020-11-23%2009%3A23%3A16): Upgrade Complete Optimizer statistics are not transferred by pg_upgrade so, once you

  1   2   3   4   5   6   7   8   9   10   >