Re: pg_dump: optimize dumpFunc()

2024-08-03 Thread Neil Conway
atch size vs client-side memory consumption. Debatable whether it is worth the hit to code readability though, and might be a bit grotty to do the tsort we need to do for dependency handling... Neil

Re: RFC: adding pytest as a supported test framework

2024-06-12 Thread FWS Neil
> On Jun 12, 2024, at 6:40 AM, Jelte Fennema-Nio wrote: > > I think C#, Java, Go, Rust, Kotlin, and Swift would be acceptable > choices for me (and possibly some more). They allow some type of > introspection, they have a garbage collector, and their general > tooling is quite good. > Having

Re: Optimizing COPY with SIMD

2024-06-07 Thread Neil Conway
493 s … 1.552 s10 runs Benchmark 1: ./psql -f /Users/neilconway/copy-from-large-short-strings.sql Time (mean ± σ): 1.032 s ± 0.032 s[User: 0.002 s, System: 0.001 s] Range (min … max):1.009 s … 1.113 s10 runs Neil v4-0005-Optimize-COPY-TO-in-text-format-using-SIMD.patc

Re: small pg_dump code cleanup

2024-06-05 Thread Neil Conway
erformance difference is noticeable. Personally I don't think the marginal performance win justifies the hit to readability, but I don't feel strongly about it. Neil

Re: Optimizing COPY with SIMD

2024-06-05 Thread Neil Conway
… max): 842.1 ms … 891.6 ms10 runs Benchmark 1: ./psql -f /Users/neilconway/copy-out-bench-text-short.sql Time (mean ± σ): 536.7 ms ± 10.9 ms[User: 1.2 ms, System: 0.0 ms] Range (min … max): 530.1 ms … 566.8 ms10 runs === Looks like there is a slight regression for sh

Re: small pg_dump code cleanup

2024-06-05 Thread Neil Conway
allocation and then index into it to fill-in each struct before entering it into the hash table. It might be more straightforward to just malloc each individual struct. Neil

Re: Optimizing COPY with SIMD

2024-06-03 Thread Neil Conway
On Mon, Jun 3, 2024 at 9:22 AM Joe Conway wrote: > Welcome back! > Thanks Joe! It's been a minute :) Neil

Optimizing COPY with SIMD

2024-06-02 Thread Neil Conway
now, the POC patch falls back to the old code path when encoding_embeds_ascii is true. Any feedback would be very welcome. Cheers, Neil [1] https://www.postgresql.org/message-id/caaphdvplxwmzvbckcdgfu9xqjgcdm7tfprdtxub9pvgpnuy...@mail.gmail.com 0002-Optimize-COPY-TO-.-FORMAT-CSV-using-SIMD-inst

Re: psql: \dl+ to list large objects privileges

2021-09-17 Thread Neil Chen
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed Hi, I think this is an interesting patch. +1 I tested it for

Re: Automatic notification of top transaction IDs

2021-07-27 Thread Neil Chen
Greetings, I simply tested it and it works well. But I got a compilation warning, should we move the definition of function FullTransactionIdToStr to the "transam.h"? -- There is no royal road to learning. HighGo Software Co.

Re: storing an explicit nonce

2021-05-27 Thread Neil Chen
On Fri, May 28, 2021 at 2:12 PM Neil Chen wrote: > > When the checksums are disabled, we can use these unused checksum values > as a counter to make sure we have different nonce even if we don't write > the new WAL record. > > Ah, well, I think I've figured it out

Re: storing an explicit nonce

2021-05-27 Thread Neil Chen
On Thu, May 27, 2021 at 11:12 PM Bruce Momjian wrote: > > Well, the code now does write full page images for hint bit changes, so > it should work fine. > > Yes, indeed it works well and I'd tested it. But here I want to make clear my understanding of the argument, if there is any problem please

Re: storing an explicit nonce

2021-05-27 Thread Neil Chen
Greetings, On Thu, May 27, 2021 at 4:52 PM Bruce Momjian wrote: > > > > I am confused why checksums, which are widely used, acceptably require > > wal_log_hints, but there is concern that file encryption, which is > > heavier, cannot acceptably require wal_log_hints. I must be missing > > somet

Re: [PATCH] In psql \?, add [+] annotation where appropriate

2021-05-24 Thread Neil Chen
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed Hi, thank you for your work. I think this is a meaningful pat

Re: Key management with tests

2021-04-06 Thread Neil Chen
Hi Bruce, I went through these patches and executed the test script you added for the KMS section, which looks all good. This is a point that looks like a bug - in patch 10, you changed the location and use of *RelFileNodeSkippingWAL()*, but the modified code logic seems different from the origin

Re: BUG #16583: merge join on tables with different DB collation behind postgres_fdw fails

2021-03-03 Thread Neil Chen
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:tested, passed Greetings, I learned about the patch and read your discussions.

Re: Phrase search vs. multi-lexeme tokens

2021-01-25 Thread Neil Chen
Hi Alexander, On Mon, Jan 25, 2021 at 11:25 PM Alexander Korotkov wrote: > > BTW, you mentioned you read the documentation. Do you think it needs > to be adjusted accordingly to the patch? > > Yes, I checked section 8.11, section 9.13 and Chapter 12 of the document. The change of this patch did

Re: Phrase search vs. multi-lexeme tokens

2021-01-25 Thread Neil Chen
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:not tested Greetings, Although I am not an expert in this field, I carefully re

Re: Key management with tests

2021-01-12 Thread Neil Chen
Thank you for your reply, On Wed, Jan 13, 2021 at 12:08 AM Stephen Frost wrote: > > No, we can't 'modify the page format as we wish'- if we change away from > using a C structure then we're going to be modifying quite a bit of > code which otherwise doesn't need to be changed. The proposed flag

Re: Key management with tests

2021-01-11 Thread Neil Chen
Hi Stephen, On Tue, Jan 12, 2021 at 10:47 AM Stephen Frost wrote: > > This is an interesting question but ultimately I don't think we should > be looking at this from the perspective of allowing arbitrary changes to > the page format. The challenge is that much of the page format, today, > is d

Re: Proposed patch for key management

2021-01-04 Thread Neil Chen
On Tue, Jan 5, 2021 at 10:18 AM Bruce Momjian wrote: > On Fri, Jan 1, 2021 at 06:26:36PM +, Alastair Turner wrote: > > There is all sorts of flexibility being proposed: > > * scope of keys > * encryption method > * encryption mode > * internal/external > > Some of this is related to wrap

Re: Proposed patch for key managment

2020-12-17 Thread Neil Chen
On Fri, Dec 18, 2020 at 3:02 AM Bruce Momjian wrote: > > Here is a run of all four authentication methods, and updated scripts. > I have renamed Yubiki to PIV since the script should work with anY > PIV-enabled deviced, like a CAC. > > Thanks for attaching these patches. The unfortunate thing is

Re: Proposed patch for key managment

2020-12-14 Thread Neil Chen
Hi, Bruce I read your question and here are some of my thoughts. Why is KmgrShmemData a struct, when it only has a single member? > Are > all shared memory areas structs? > Yes, all areas created by ShmemInitStruct() are structs. I think the significance of using struct is that i

Re: Proposed patch for key managment

2020-12-10 Thread Neil Chen
Hi, everyone > > I have read the patch and did some simple tests. I'm not entirely sure > about some code segments; e.g.: > > In the BootStrapKmgr() we generate a data encryption key by: > key = generate_crypto_key(file_encryption_keylen); > > However, I found that the file_encryption_keylen is alw

Re: BUG #16663: DROP INDEX did not free up disk space: idle connection hold file marked as deleted

2020-11-19 Thread Neil Chen
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:not tested Hi, I have tested the feature and it worked well. One thing that doe

Re: Support for DATETIMEOFFSET

2020-04-11 Thread Neil
> On Apr 10, 2020, at 6:10 PM, Jeremy Morton wrote: > > Neil wrote: >>> On Apr 10, 2020, at 8:19 AM, Jeremy Morton wrote: >>> >>> Oh well. Guess I keep using SQL Server then. datetimeoffset makes it >>> impossible for developers to make the mis

Re: Support for DATETIMEOFFSET

2020-04-10 Thread Neil
lt, just trying to understand. Neil > > -- > Best regards, > Jeremy Morton (Jez) > > Andreas Karlsson wrote: >> On 4/10/20 10:34 AM, Jeremy Morton wrote: >>> I've noticed that Postgres doesn't have support for DATETIMEOFFSET (or any >>> func