Re: Test 002_pg_upgrade fails with olddump on Windows

06.12.2023 04:17, Michael Paquier wrote: At the end, just applying the filtering all the time makes the most sense to me, so I've applied a patch doing just that. Thank you for the fix! Now that test with the minimal dump passes fine, but when I tried to run it with a complete dump borrowed fr

Re: tablecmds.c/MergeAttributes() cleanup

On 05.10.23 17:49, Peter Eisentraut wrote: On 19.09.23 15:11, Peter Eisentraut wrote: Here is an updated version of this patch set.  I resolved some conflicts and addressed this comment of yours.  I also dropped the one patch with some catalog header edits that people didn't seem to particular

RE: Partial aggregates pushdown

Hi Mr.Haas, hackers. > From: Robert Haas > Sent: Tuesday, November 28, 2023 5:03 AM > Also, I want to make one other point here about security and reliability. > Right now, there is no way for a user to feed > arbitrary data to a deserialization function. Since serialization and > deserializati

Re: logical decoding and replication of sequences, take 2

On Tue, Dec 5, 2023 at 10:23 PM Tomas Vondra wrote: > > On 12/5/23 13:17, Amit Kapila wrote: > > > (b) for transactional > > cases, we see overhead due to traversing all the top-level txns and > > check the hash table for each one to find whether change is > > transactional. > > > > Not really, no

Re: SQL:2011 application time

On 02.12.23 19:41, Paul Jungwirth wrote: So what do you think of this idea instead?: We could add a new (optional) support function to GiST that translates "well-known" strategy numbers into the opclass's own strategy numbers. This would be support function 12. Then we can say translateStrate

Re: logical decoding and replication of sequences, take 2

On Wed, Dec 6, 2023 at 11:12 AM Dilip Kumar wrote: > > On Sun, Dec 3, 2023 at 11:22 PM Tomas Vondra > wrote: > > I was also wondering what happens if the sequence changes are transactional but somehow the snap builder state changes to SNAPBUILD_FULL_SNAPSHOT in between processing of the smgr_dec

Re: Clean up some signal usage mainly related to Windows

On 04.12.23 18:20, Tristan Partin wrote: On Mon Dec 4, 2023 at 9:22 AM CST, Peter Eisentraut wrote: On 01.12.23 23:10, Tristan Partin wrote: > On Wed Jul 12, 2023 at 9:35 AM CDT, Tristan Partin wrote: >> On Wed Jul 12, 2023 at 9:31 AM CDT, Peter Eisentraut wrote: >> > On 12.07.23 16:23, Tristan

Re: Synchronizing slots from primary to standby

Hi, On 12/6/23 7:18 AM, shveta malik wrote: On Wed, Dec 6, 2023 at 10:56 AM Amit Kapila wrote: I feel that is indirectly relying on the fact that the primary won't advance logical slots unless physical standby has consumed data. Yes, that is the basis of this discussion. Yes. But now on

Re: RFI: Extending the TOAST Pointer

Hi, Here's the PoC for a custom TOAST pointer. The main idea is that custom pointer provides data space allowing to store custom metadata (i.e. TOAST method, relation OIDs, advanced compression information, etc, and even keep part of the data inline. Any feedback would be greatly appreciated. --

Re: logical decoding and replication of sequences, take 2

On Wed, Dec 6, 2023 at 11:12 AM Dilip Kumar wrote: > > On Sun, Dec 3, 2023 at 11:22 PM Tomas Vondra > wrote: > > > > > Some time ago I floated the idea of maybe "queuing" the sequence changes > > and only replay them on the next commit, somehow. But we did ran into > > problems with which snapsho

Re: logical decoding and replication of sequences, take 2

On Sun, Dec 3, 2023 at 11:56 PM Tomas Vondra wrote: > > On 12/3/23 18:52, Tomas Vondra wrote: > > ... > > > > Another idea is that maybe we could somehow inform ReorderBuffer whether > the output plugin even is interested in sequences. That'd help with > cases where we don't even want/need to repl

Re: remaining sql/json patches

On 2023-Dec-05, Amit Langote wrote: > I've attempted to trim down the JSON_TABLE grammar (0004), but this is > all I've managed so far. Among other things, I couldn't refactor the > grammar to do away with the following: > > +%nonassoc NESTED > +%left PATH To recap, the reason we're argui

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

On Tue, Dec 5, 2023 at 6:07 PM Alena Rybakina wrote: > > Hi! > > Thank you for your contribution to this thread. > > > I reviewed it and have a few questions. > > 1. I have seen that you delete a table before creating it, to which you want > to add errors due to a failed "copy from" operation. I

Re: Synchronizing slots from primary to standby

On Wed, Dec 6, 2023 at 3:00 PM Drouvot, Bertrand wrote: > > Hi, > > On 12/6/23 7:18 AM, shveta malik wrote: > > On Wed, Dec 6, 2023 at 10:56 AM Amit Kapila wrote: > >> > >> I feel that is indirectly relying on the fact that the primary won't > >> advance logical slots unless physical standby has

Re: Row pattern recognition

On 04.12.23 12:40, Tatsuo Ishii wrote: diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index d631ac89a9..5a77fca17f 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -251,6 +251,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Nod

Re: logical decoding and replication of sequences, take 2

On Wed, Dec 6, 2023 at 3:36 PM Amit Kapila wrote: > > > Why can't we use the same concept of > > SnapBuildDistributeNewCatalogSnapshot(), I mean we keep queuing the > > non-transactional changes (have some base snapshot before the first > > change), and whenever there is any catalog change, queue

Re: Is WAL_DEBUG related code still relevant today?

On 02.12.23 15:06, Bharath Rupireddy wrote: I enabled this code by compiling with the WAL_DEBUG macro and setting wal_debug GUC to on. Firstly, the compilation on Windows failed because XL_ROUTINE was passed inappropriately for XLogReaderAllocate() used. This kind of thing could be mostly avoid

Re: Make COPY format extendable: Extract COPY TO format implementations

Sutou Kouhei wrote: > * 2022-04: Apache Arrow [2] > * 2018-02: Apache Avro, Apache Parquet and Apache ORC [3] > > (FYI: I want to add support for Apache Arrow.) > > There were discussions how to add support for more formats. [3][4] > In these discussions, we got a consensus about making

Re: Emitting JSON to file using COPY TO

On 2023-12-05 Tu 16:46, Joe Conway wrote: On 12/5/23 16:20, Andrew Dunstan wrote: On 2023-12-05 Tu 16:09, Joe Conway wrote: On 12/5/23 16:02, Joe Conway wrote: On 12/5/23 15:55, Andrew Dunstan wrote: and in any other case (e.g. LINES) I can't see why you would have them. Oh I didn't addre

Re: Is WAL_DEBUG related code still relevant today?

On Wed, Dec 6, 2023, at 8:27 AM, Peter Eisentraut wrote: > On 02.12.23 15:06, Bharath Rupireddy wrote: > > I enabled this code by compiling with the WAL_DEBUG macro and setting > > wal_debug GUC to on. Firstly, the compilation on Windows failed > > because XL_ROUTINE was passed inappropriately for

Forbid the use of invalidated physical slots in streaming replication.

Hi, When testing streaming replication with a physical slot. I found an unexpected behavior that the walsender could use an invalidated physical slot for streaming. This occurs when the primary slot is invalidated due to reaching the max_slot_wal_keep_size before initializing the streaming replic

Add missing error codes to PANIC/FATAL error reports in xlog.c and relcache.c

Hi, There is an ongoing thread [1] for adding missing SQL error codes to PANIC and FATAL error reports in xlogrecovery.c file. I did the same but for xlog.c and relcache.c files. I couldn't find a suitable error code for the "cache lookup failed for relation" error in relcache.c and this error co

Re: Transaction timeout

> On 30 Nov 2023, at 20:06, Andrey M. Borodin wrote: > > > Tomorrow I plan to fix raising of the timeout when the transaction is idle. > Renaming transaction_timeout to something else (to avoid confusion with > prepared xacts) also seems correct to me. Here's a v6 version of the feature. Ch

Re: Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

On Tue, Dec 5, 2023 at 8:15 PM Peter Geoghegan wrote: > Just to be clear, you're raising a concern that seems to me to apply > to "the other optimization" from the same commit, specifically -- the > precheck optimization. Not the one I found a problem in. (They're > closely related but distinct op

Re: Possible segfault when sending notification within a ProcessUtility hook

> On Tue, Dec 5, 2023 at 9:03 PM Tom Lane wrote: > Why should we regard that as anything other than a bug in the > ProcessUtility hook? A failed transaction should not send any > notifies. Fair point. That was also my initial assumption but I thought that the transaction state was not available

Re: SQL:2011 application time

On Sun, Dec 3, 2023 at 2:11 AM Paul Jungwirth wrote: > > v19 patch series attached, rebased to a11c9c42ea. > this TODO: * TODO: It sounds like FOR PORTION OF might need to do something here too? based on comments on ExprContext. I refactor a bit, and solved this TODO. tring to the following TOD

Re: Partial aggregates pushdown

On Wed, Dec 6, 2023 at 3:41 AM fujii.y...@df.mitsubishielectric.co.jp wrote: > Are you concerned about the hassle and potential human errors of manually > adding new partial > aggregation functions, rather than the catalog becoming bloated? I'm concerned about both. > The process of creating pa

Re: Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

On Wed, 6 Dec 2023 at 14:11, Robert Haas wrote: > > On Tue, Dec 5, 2023 at 8:15 PM Peter Geoghegan wrote: > > Just to be clear, you're raising a concern that seems to me to apply > > to "the other optimization" from the same commit, specifically -- the > > precheck optimization. Not the one I fou

Re: Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

On Wed, Dec 6, 2023 at 8:27 AM Matthias van de Meent wrote: > I feel that Peter refered to these two distinct optimizations: > > 1. When scanning an index in ascending order using scankey a > 1 (so, > one that defines a start point of the scan), we don't need to check > items for consistency with

Re: logical decoding and replication of sequences, take 2

On 12/6/23 10:05, Dilip Kumar wrote: > On Wed, Dec 6, 2023 at 11:12 AM Dilip Kumar wrote: >> >> On Sun, Dec 3, 2023 at 11:22 PM Tomas Vondra >> wrote: >>> > > I was also wondering what happens if the sequence changes are > transactional but somehow the snap builder state changes to > SNAPBUILD_F

Re: logical decoding and replication of sequences, take 2

On 12/6/23 12:05, Dilip Kumar wrote: > On Wed, Dec 6, 2023 at 3:36 PM Amit Kapila wrote: >> >>> Why can't we use the same concept of >>> SnapBuildDistributeNewCatalogSnapshot(), I mean we keep queuing the >>> non-transactional changes (have some base snapshot before the first >>> change), and when

Re: Emitting JSON to file using COPY TO

On 12/6/23 07:36, Andrew Dunstan wrote: On 2023-12-05 Tu 16:46, Joe Conway wrote: On 12/5/23 16:20, Andrew Dunstan wrote: On 2023-12-05 Tu 16:09, Joe Conway wrote: On 12/5/23 16:02, Joe Conway wrote: On 12/5/23 15:55, Andrew Dunstan wrote: and in any other case (e.g. LINES) I can't see why

Re: logical decoding and replication of sequences, take 2

On 12/6/23 11:19, Amit Kapila wrote: > On Sun, Dec 3, 2023 at 11:56 PM Tomas Vondra > wrote: >> >> On 12/3/23 18:52, Tomas Vondra wrote: >>> ... >>> >> >> Another idea is that maybe we could somehow inform ReorderBuffer whether >> the output plugin even is interested in sequences. That'd help with

Re: Make COPY format extendable: Extract COPY TO format implementations

On Wed, Dec 6, 2023 at 3:28 PM Sutou Kouhei wrote: > > Hi, > > In > "Re: Make COPY format extendable: Extract COPY TO format implementations" > on Wed, 6 Dec 2023 15:11:34 +0800, > Junwang Zhao wrote: > > > For the extra curly braces, I mean the following code block in > > CopyToFormatBinar

Re: logical decoding and replication of sequences, take 2

On 12/6/23 09:56, Amit Kapila wrote: > On Tue, Dec 5, 2023 at 10:23 PM Tomas Vondra > wrote: >> >> On 12/5/23 13:17, Amit Kapila wrote: >> >>> (b) for transactional >>> cases, we see overhead due to traversing all the top-level txns and >>> check the hash table for each one to find whether change

Re: Make COPY format extendable: Extract COPY TO format implementations

On Wed, Dec 6, 2023 at 8:32 PM Daniel Verite wrote: > > Sutou Kouhei wrote: > > > * 2022-04: Apache Arrow [2] > > * 2018-02: Apache Avro, Apache Parquet and Apache ORC [3] > > > > (FYI: I want to add support for Apache Arrow.) > > > > There were discussions how to add support for more form

Re: Is WAL_DEBUG related code still relevant today?

Peter Eisentraut writes: > This kind of thing could be mostly avoided if we didn't hide all the > WAL_DEBUG behind #ifdefs. For example, in the attached patch, I instead > changed it so that > if (XLOG_DEBUG) > resolves to > if (false) > in the normal case. That way, we don't need to

Re: [RFC] Clang plugin for catching suspicious typedef casting

> On Sun, Dec 03, 2023 at 07:02:55PM -0800, Peter Geoghegan wrote: > On Sun, Dec 3, 2023 at 6:31 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > Only advantage I see to using libclang is that you can run programs built > > > with libclang regardless of what your C compiler is. I typically use

Re: remaining sql/json patches

On 2023-Dec-06, Amit Langote wrote: > I think I'm inclined toward adapting the LA-token fix (attached 0005), > because we've done that before with SQL/JSON constructors patch. > Also, if I understand the concerns that Tom mentioned at [1] > correctly, maybe we'd be better off not assigning precede

Re: Emitting JSON to file using COPY TO

On 2023-12-06 We 08:49, Joe Conway wrote: On 12/6/23 07:36, Andrew Dunstan wrote: On 2023-12-05 Tu 16:46, Joe Conway wrote: On 12/5/23 16:20, Andrew Dunstan wrote: On 2023-12-05 Tu 16:09, Joe Conway wrote: On 12/5/23 16:02, Joe Conway wrote: On 12/5/23 15:55, Andrew Dunstan wrote: and in

Re: Emitting JSON to file using COPY TO

Joe Conway writes: > I believe this is ready to commit unless there are further comments or > objections. I thought we were still mostly at proof-of-concept stage? In particular, has anyone done any performance testing? I'm concerned about that because composite_to_json() has zero capability to

Re: Emitting JSON to file using COPY TO

On 12/6/23 10:32, Andrew Dunstan wrote: On 2023-12-06 We 08:49, Joe Conway wrote: On 12/6/23 07:36, Andrew Dunstan wrote: On 2023-12-05 Tu 16:46, Joe Conway wrote: On 12/5/23 16:20, Andrew Dunstan wrote: On 2023-12-05 Tu 16:09, Joe Conway wrote: On 12/5/23 16:02, Joe Conway wrote: On 12/5

Re: Clean up some signal usage mainly related to Windows

On Wed, Dec 06, 2023 at 10:23:52AM +0100, Peter Eisentraut wrote: > Ok, I have committed your 0001 patch. My compiler is unhappy about this one: ../postgresql/src/bin/pg_test_fsync/pg_test_fsync.c:605:2: error: ignoring return value of ‘write’, declared with attribute warn_unused_result [-Werro

Re: Emitting JSON to file using COPY TO

On 12/6/23 10:44, Tom Lane wrote: Joe Conway writes: I believe this is ready to commit unless there are further comments or objections. I thought we were still mostly at proof-of-concept stage? The concept is narrowly scoped enough that I think we are homing in on the final patch. In pa

Re: Emitting JSON to file using COPY TO

On 2023-12-06 We 10:44, Tom Lane wrote: Joe Conway writes: I believe this is ready to commit unless there are further comments or objections. I thought we were still mostly at proof-of-concept stage? In particular, has anyone done any performance testing? I'm concerned about that because co

Re: Emitting JSON to file using COPY TO

Joe Conway writes: > On 12/6/23 10:44, Tom Lane wrote: >> In particular, has anyone done any performance testing? > I will devise some kind of test and report back. I suppose something > with many rows and many narrow columns comparing time to COPY > text/csv/json modes would do the trick? Yea

Re: Remove MSVC scripts from the tree

On 2023-12-06 We 01:18, Peter Eisentraut wrote: On 04.12.23 21:11, Andrew Dunstan wrote: I just had a look at shifting bowerbird to use meson, and it got stymied at the c99 test, which apparently doesn't compile with anything less than VS2019. If that is the case, then wouldn't that invalid

Re: Emitting JSON to file using COPY TO

Big +1 to this overall feature. This is something I've wanted for a long time as well. While it's possible to use a COPY with text output for a trivial case, the double escaping falls apart quickly for arbitrary data. It's really only usable when you know exactly what you are querying and know it

Re: Clean up some signal usage mainly related to Windows

On Wed Dec 6, 2023 at 10:18 AM CST, Nathan Bossart wrote: On Wed, Dec 06, 2023 at 10:23:52AM +0100, Peter Eisentraut wrote: > Ok, I have committed your 0001 patch. My compiler is unhappy about this one: ../postgresql/src/bin/pg_test_fsync/pg_test_fsync.c:605:2: error: ignoring return value of

Re: Emitting JSON to file using COPY TO

Andrew Dunstan writes: > On 2023-12-06 We 10:44, Tom Lane wrote: >> In particular, has anyone done any performance testing? >> I'm concerned about that because composite_to_json() has >> zero capability to cache any metadata across calls, meaning >> there is going to be a large amount of duplicate

Re: Emitting JSON to file using COPY TO

On Wed, Dec 06, 2023 at 11:28:59AM -0500, Tom Lane wrote: > It might be acceptable to plan on improving the performance later, > depending on just how bad it is now. On 10M rows with 11 integers each, I'm seeing the following: (format text) Time: 10056.311 ms (00:10.056) T

Re: Clean up some signal usage mainly related to Windows

On Wed, Dec 06, 2023 at 10:28:49AM -0600, Tristan Partin wrote: > According to my setup, I am hitting the /* Ignore */ variant of __wur. I am > guessing that Fedora doesn't add fortification to the default CFLAGS. What > distro are you using? But yes, something like what you proposed sounds good >

Re: Emitting JSON to file using COPY TO

On Wed, Dec 06, 2023 at 10:33:49AM -0600, Nathan Bossart wrote: > (format csv) > Time: 12295.480 ms (00:12.295) > Time: 12311.059 ms (00:12.311) > Time: 12305.469 ms (00:12.305) > > (format json) > Time: 24568.621 ms (00:24.569) > Time: 23756.234 ms (00:23

Re: Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

On Wed, Dec 6, 2023 at 5:27 AM Matthias van de Meent wrote: > On Wed, 6 Dec 2023 at 14:11, Robert Haas wrote: > > It isn't very clear from the commit message that this commit is doing > > two different things, and in fact I'm still unclear on what exactly > > the other optimization is. > > I feel

Re: Change GUC hashtable to use simplehash?

On Wed, 2023-12-06 at 07:39 +0700, John Naylor wrote: > "git grep cstring_hash" found nothing, so not sure what you're > asking. Sorry, I meant string_hash(). Your v5-0002 changes the way hashing works for cstrings, and that means it's no longer equivalent to hash_bytes with strlen. That's probabl

Re: Remove MSVC scripts from the tree

On 06.12.23 17:27, Andrew Dunstan wrote: But the VS2019 compiler implements enough of C99 to pass our meson test, unlike VS2017. Maybe the test is too strict. After all, we know we can in fact build with the earlier versions. I just realized that the C99 test is actually our own, not provided

Re: Clean up some signal usage mainly related to Windows

On 06.12.23 17:18, Nathan Bossart wrote: On Wed, Dec 06, 2023 at 10:23:52AM +0100, Peter Eisentraut wrote: Ok, I have committed your 0001 patch. My compiler is unhappy about this one: ../postgresql/src/bin/pg_test_fsync/pg_test_fsync.c:605:2: error: ignoring return value of ‘write’, declared

Re: Clean up some signal usage mainly related to Windows

On Wed, Dec 06, 2023 at 06:27:04PM +0100, Peter Eisentraut wrote: > Makes sense. Can you commit that? Yes, I will do so shortly. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

On Tue, Dec 5, 2023 at 8:20 PM Peter Geoghegan wrote: > On Tue, Dec 5, 2023 at 8:06 PM Alexander Korotkov > wrote: > > Thank you for raising this issue. Preprocessing of btree scan keys is > > normally removing the redundant scan keys. However, redundant scan > > keys aren't removed when they

Re: Building PosgresSQL with LLVM fails on Solaris 11.4

Hi, On 2023-12-01 23:06:59 +, Sacha Hottinger wrote: > // I used the patch command to patch the src/backend/port/Makefile with your > attached file and tried again with the Sun Studio compiler. There is now a > different error at this stage: > … > /opt/developerstudio12.6/bin/cc -m64 -xarch=

Re: generic plans and "initial" pruning

Reviewing 0001: Perhaps ExecEndCteScan needs an adjustment. What if node->leader was never set? Other than that, I think this is in good shape. Maybe there are other things we'd want to adjust here, or maybe there aren't, but there doesn't seem to be any good reason to bundle more changes into th

Re: Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

On Wed, Dec 6, 2023 at 5:27 AM Matthias van de Meent wrote: > 1. When scanning an index in ascending order using scankey a > 1 (so, > one that defines a start point of the scan), we don't need to check > items for consistency with that scankey once we've found the first > value that is consistent

Re: Emitting JSON to file using COPY TO

Andrew Dunstan wrote: > IMNSHO, we should produce either a single JSON > document (the ARRAY case) or a series of JSON documents, one per row > (the LINES case). "COPY Operations" in the doc says: " The backend sends a CopyOutResponse message to the frontend, followed by zero or mor

Re: Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

On Wed, 6 Dec 2023 at 19:55, Peter Geoghegan wrote: > > On Wed, Dec 6, 2023 at 5:27 AM Matthias van de Meent > wrote: > > 1. When scanning an index in ascending order using scankey a > 1 (so, > > one that defines a start point of the scan), we don't need to check > > items for consistency with th

Re: Emitting JSON to file using COPY TO

On 12/6/23 13:59, Daniel Verite wrote: Andrew Dunstan wrote: IMNSHO, we should produce either a single JSON document (the ARRAY case) or a series of JSON documents, one per row (the LINES case). "COPY Operations" in the doc says: " The backend sends a CopyOutResponse message to the

Re: Emitting JSON to file using COPY TO

On 12/6/23 11:44, Nathan Bossart wrote: On Wed, Dec 06, 2023 at 10:33:49AM -0600, Nathan Bossart wrote: (format csv) Time: 12295.480 ms (00:12.295) Time: 12311.059 ms (00:12.311) Time: 12305.469 ms (00:12.305) (format json) Time: 24568.621 ms (00:

Re: Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

On Wed, Dec 6, 2023 at 11:14 AM Matthias van de Meent wrote: > I was thinking more along the lines of page splits+deletions while > we're doing _bt_stepright(), but forgot to consider that we first lock > the right sibling, and only then release the left sibling for splits, > so we should be fine

automating RangeTblEntry node support

I have been looking into what it would take to get rid of the custom_read_write and custom_query_jumble for the RangeTblEntry node type. This is one of the larger and more complex exceptions left. (Similar considerations would also apply to the Constraint node type.) Allegedly, only certain f

Re: Emitting JSON to file using COPY TO

Joe Conway writes: > I'll see if I can add some caching to composite_to_json(), but based on > the relative data size it does not sound like there is much performance > left on the table to go after, no? If Nathan's perf results hold up elsewhere, it seems like some micro-optimization around th

Re: gai_strerror() is not thread-safe on Windows

On Tue, Dec 5, 2023 at 3:43 PM Kyotaro Horiguchi wrote: > At Tue, 5 Dec 2023 08:26:54 +1300, Thomas Munro > wrote in > > On second thoughts, I guess it would make more sense to use the exact > > messages Windows' own implementation would return instead of whatever > > we had in the past (probabl

Re: Remove MSVC scripts from the tree

On 2023-12-06 We 12:24, Peter Eisentraut wrote: On 06.12.23 17:27, Andrew Dunstan wrote: But the VS2019 compiler implements enough of C99 to pass our meson test, unlike VS2017. Maybe the test is too strict. After all, we know we can in fact build with the earlier versions. I just realized t

Re: Emitting JSON to file using COPY TO

On 2023-12-06 We 15:20, Tom Lane wrote: Joe Conway writes: I'll see if I can add some caching to composite_to_json(), but based on the relative data size it does not sound like there is much performance left on the table to go after, no? If Nathan's perf results hold up elsewhere, it seems l

Reducing output size of nodeToString

Hi, PFA a patch that reduces the output size of nodeToString by 50%+ in most cases (measured on pg_rewrite), which on my system reduces the total size of pg_rewrite by 33% to 472KiB. This does keep the textual pg_node_tree format alive, but reduces its size signficantly. The basic techniques used

Re: automating RangeTblEntry node support

On Wed, 6 Dec 2023 at 21:02, Peter Eisentraut wrote: > > I have been looking into what it would take to get rid of the > custom_read_write and custom_query_jumble for the RangeTblEntry node > type. This is one of the larger and more complex exceptions left. > [...] > Now one could probably rightf

Re: Emitting JSON to file using COPY TO

On 12/6/23 11:28, Sehrope Sarkuni wrote: Big +1 to this overall feature. cool! Regarding the defaults for the output, I think JSON lines (rather than a JSON array of objects) would be preferred. It's more natural to combine them and generate that type of data on the fly rather than forcing

Re: Emitting JSON to file using COPY TO

On Wed, Dec 6, 2023 at 4:03 PM Andrew Dunstan wrote: > > The output size difference does say that maybe we should pay some > > attention to the nearby request to not always label every field. > > Perhaps there should be an option for each row to transform to > > a JSON array rather than an object

Re: Emitting JSON to file using COPY TO

On Wed, Dec 06, 2023 at 03:20:46PM -0500, Tom Lane wrote: > If Nathan's perf results hold up elsewhere, it seems like some > micro-optimization around the text-pushing (appendStringInfoString) > might be more useful than caching. The 7% spent in cache lookups > could be worth going after later, bu

Re: Emitting JSON to file using COPY TO

On Wed, Dec 6, 2023 at 4:29 PM Joe Conway wrote: > > 1. Outputting a top level JSON object without the additional column > > keys. IIUC, the top level keys are always the column names. A common use > > case would be a single json/jsonb column that is already formatted > > exactly as the user woul

Re: Emitting JSON to file using COPY TO

On 12/6/23 16:42, Sehrope Sarkuni wrote: On Wed, Dec 6, 2023 at 4:29 PM Joe Conway > wrote: > 1. Outputting a top level JSON object without the additional column > keys. IIUC, the top level keys are always the column names. A common use > case would

Re: Emitting JSON to file using COPY TO

On Wed, Dec 6, 2023 at 3:38 PM Joe Conway wrote: > So the questions are: > 1. Do those two formats work for the initial implementation? > Yes. We provide a stream-oriented format and one atomic-import format. 2. Is the default correct or should it be switched > e.g. rather than specifying

Re: Emitting JSON to file using COPY TO

On 12/6/23 14:47, Joe Conway wrote: On 12/6/23 13:59, Daniel Verite wrote: Andrew Dunstan wrote: IMNSHO, we should produce either a single JSON document (the ARRAY case) or a series of JSON documents, one per row (the LINES case). "COPY Operations" in the doc says: " The backend se

Re: Clean up some signal usage mainly related to Windows

On Wed, Dec 06, 2023 at 11:30:02AM -0600, Nathan Bossart wrote: > On Wed, Dec 06, 2023 at 06:27:04PM +0100, Peter Eisentraut wrote: >> Makes sense. Can you commit that? > > Yes, I will do so shortly. Committed. Apologies for the delay. -- Nathan Bossart Amazon Web Services: https://aws.amazon

Re: Emitting JSON to file using COPY TO

On Wed, Dec 6, 2023 at 4:09 PM Joe Conway wrote: > On 12/6/23 14:47, Joe Conway wrote: > > On 12/6/23 13:59, Daniel Verite wrote: > >> Andrew Dunstan wrote: > >> > >>> IMNSHO, we should produce either a single JSON > >>> document (the ARRAY case) or a series of JSON documents, one per row >

Re: PATCH: Add REINDEX tag to event triggers

On Wed, Dec 06, 2023 at 10:00:01AM +0300, Alexander Lakhin wrote: > I agree with it. I had worried a bit about ReindexRelationConcurrently() > becoming twofold for callers (it can leave the snapshot or pop it), but I > couldn't find a way to hide this twofoldness inside without adding more > comple

Re: RFC: Logging plan of the running query

Hello hackers, Last Saturday I submitted a patch to the pgsql-hackers list with the title "Proposal: In-flight explain logging" with a patch proposing a feature very similar to the one being worked on in this thread. I should have done a better search in the commitfest before implementing somethin

Re: Emitting JSON to file using COPY TO

On Wed, Dec 6, 2023 at 4:28 PM David G. Johnston wrote: > On Wed, Dec 6, 2023 at 4:09 PM Joe Conway wrote: > >> On 12/6/23 14:47, Joe Conway wrote: >> > On 12/6/23 13:59, Daniel Verite wrote: >> >> Andrew Dunstan wrote: >> >> >> >>> IMNSHO, we should produce either a single JSON >> >>> docu

Re: Emitting JSON to file using COPY TO

On 12/6/23 18:28, David G. Johnston wrote: On Wed, Dec 6, 2023 at 4:09 PM Joe Conway > wrote: On 12/6/23 14:47, Joe Conway wrote: > On 12/6/23 13:59, Daniel Verite wrote: >>      Andrew Dunstan wrote: >> >>> IMNSHO, we should produce either a si

Re: Emitting JSON to file using COPY TO

On 12/6/23 18:38, David G. Johnston wrote: On Wed, Dec 6, 2023 at 4:28 PM David G. Johnston mailto:david.g.johns...@gmail.com>> wrote: On Wed, Dec 6, 2023 at 4:09 PM Joe Conway mailto:m...@joeconway.com>> wrote: On 12/6/23 14:47, Joe Conway wrote: > On 12/6/23 13:59, Danie

pg16 && GSSAPI && Heimdal/Macos

Earlier this year, there was a thread about GSSAPI for delegated credentials and various operating systems ultimately that Heimdal had atrophied enough that you were comfortable not supporting it anymore as a GSSAPI library. Thread: https://www.postgresql.org/message-id/flat/ZDFTailRZzyGdbXl%40tam

RE: [CAUTION!! freemail] Re: Partial aggregates pushdown

Hi Mr.Haas. > -Original Message- > From: Robert Haas > Sent: Wednesday, December 6, 2023 10:25 PM > On Wed, Dec 6, 2023 at 3:41 AM fujii.y...@df.mitsubishielectric.co.jp > wrote: > > Are you concerned about the hassle and potential human errors of > > manually adding new partial aggregat

Re: reindexing an invalid index should not use ERRCODE_INDEX_CORRUPTED

On Wed, Dec 06, 2023 at 03:17:12PM +0900, Michael Paquier wrote: > On Sat, Nov 18, 2023 at 04:32:36PM -0800, Noah Misch wrote: > > On Sat, Nov 18, 2023 at 03:09:58PM -0800, Andres Freund wrote: > >> Unfortunately, there is a case of such an sqlstate that's not at all > >> indicating > >> corruptio

Re: Make COPY format extendable: Extract COPY TO format implementations

On Wed, Dec 06, 2023 at 10:07:51PM +0800, Junwang Zhao wrote: > I read the thread[1] you posted and I think Andres's suggestion sounds great. > > Should we extract both *copy to* and *copy from* for the first step, in that > case we can add the pg_copy_handler catalog smoothly later. > > Attached

Re: Emitting JSON to file using COPY TO

On Wed, Dec 6, 2023 at 4:45 PM Joe Conway wrote: > > " The backend sends a CopyOutResponse message to the frontend, followed > by zero or more CopyData messages (always one per row), followed by > CopyDone" > > probably "always one per row" would be changed to note that json array > forma

Re: Is WAL_DEBUG related code still relevant today?

On Wed, Dec 06, 2023 at 09:46:09AM -0300, Euler Taveira wrote: > On Wed, Dec 6, 2023, at 8:27 AM, Peter Eisentraut wrote: >> This kind of thing could be mostly avoided if we didn't hide all the >> WAL_DEBUG behind #ifdefs. > > AFAICS LOCK_DEBUG also hides its GUCs behind #ifdefs. The fact that XL

Re: Emitting JSON to file using COPY TO

On 12/6/23 19:39, David G. Johnston wrote: On Wed, Dec 6, 2023 at 4:45 PM Joe Conway > wrote: " The backend sends a CopyOutResponse message to the frontend, followed     by zero or more CopyData messages (always one per row), followed by     CopyDone"

Re: Emitting JSON to file using COPY TO

On Wed, Dec 6, 2023 at 5:57 PM Joe Conway wrote: > On 12/6/23 19:39, David G. Johnston wrote: > > On Wed, Dec 6, 2023 at 4:45 PM Joe Conway > > wrote: > > > But I still cannot shake the belief that using a format code of 1 - > > which really could be interpreted as mea

Re: Emitting JSON to file using COPY TO

On 12/6/23 18:09, Joe Conway wrote: On 12/6/23 14:47, Joe Conway wrote: On 12/6/23 13:59, Daniel Verite wrote: Andrew Dunstan wrote: IMNSHO, we should produce either a single JSON document (the ARRAY case) or a series of JSON documents, one per row (the LINES case). "COPY Operation

Re: Emitting JSON to file using COPY TO

On 12/6/23 20:09, David G. Johnston wrote: On Wed, Dec 6, 2023 at 5:57 PM Joe Conway > wrote: On 12/6/23 19:39, David G. Johnston wrote: > On Wed, Dec 6, 2023 at 4:45 PM Joe Conway mailto:m...@joeconway.com> >

Re: Emitting JSON to file using COPY TO

On Wed, Dec 6, 2023 at 6:14 PM Joe Conway wrote: > > > But the point that we should introduce a 2 still stands. The new code > > would mean: use text output functions but that there is no inherent > > tabular structure in the underlying contents. Instead the copy format > > was JSON and the out

Re: reindexing an invalid index should not use ERRCODE_INDEX_CORRUPTED

On Wed, Dec 06, 2023 at 04:33:33PM -0800, Noah Misch wrote: > On Wed, Dec 06, 2023 at 03:17:12PM +0900, Michael Paquier wrote: >>> The "cannot" part of the message is also inaccurate, and it's not clear to >>> me >>> why we have this specific restriction at all. REINDEX INDEX CONCURRENTLY >>> acc

  1   2   >