Re: Making aggregate deserialization (and WAL receive) functions slightly faster

2023-02-11 Thread Tom Lane
David Rowley writes: > While working on 16fd03e95, I noticed that in each aggregate > deserialization function, in order to "receive" the bytea value that > is the serialized aggregate state, appendBinaryStringInfo is used to > append the bytes of the bytea value onto a temporary StringInfoData. >

Making aggregate deserialization (and WAL receive) functions slightly faster

2023-02-11 Thread David Rowley
While working on 16fd03e95, I noticed that in each aggregate deserialization function, in order to "receive" the bytea value that is the serialized aggregate state, appendBinaryStringInfo is used to append the bytes of the bytea value onto a temporary StringInfoData. Using appendBinaryStringInfo s

Re: Importing pg_bsd_indent into our source tree

2023-02-11 Thread Thomas Munro
On Sun, Feb 12, 2023 at 2:44 PM Tom Lane wrote: > I wrote: > > Hmmm ... ci autoconf build is now happy, but the Windows run complains > > that none of the output files match. I'm betting that this is a > > Windows-newline problem, since I now see that indent.c opens both the > > input and output

Re: Importing pg_bsd_indent into our source tree

2023-02-11 Thread Tom Lane
Hmmm ... ci autoconf build is now happy, but the Windows run complains that none of the output files match. I'm betting that this is a Windows-newline problem, since I now see that indent.c opens both the input and output files in default (text) mode. I'm inclined to change it to open the output

Re: Importing pg_bsd_indent into our source tree

2023-02-11 Thread Tom Lane
Andres Freund writes: > # Running: pg_bsd_indent --version > Command 'pg_bsd_indent' not found in > /tmp/cirrus-ci-build/tmp_install/usr/local/pgsql/bin, > /tmp/cirrus-ci-build/src/tools/pg_bsd_indent, /usr/local/sbin, > /usr/local/bin, /usr/sbin, /usr/bin, /sbin, /bin at > /tmp/cirrus-ci-buil

Re: Importing pg_bsd_indent into our source tree

2023-02-11 Thread Andres Freund
Hi, On 2023-02-11 18:54:00 -0500, Tom Lane wrote: > I ended up converting the test infrastructure into a TAP test, > which kind of feels like overkill; but the Meson system doesn't > seem to provide any lower-overhead way to run a test. FWIW, The default way to indicate failures in a test is the

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2023-02-11 Thread Andres Freund
Hi, On 2022-11-01 08:32:48 +0530, Bharath Rupireddy wrote: > +/* > + * pg_pwrite_zeros > + * > + * Writes zeros to a given file. Input parameters are "fd" (file descriptor > of > + * the file), "size" (size of the file in bytes). > + * > + * On failure, a negative value is returned and errno is s

Re: refactoring relation extension and BufferAlloc(), faster COPY

2023-02-11 Thread Andres Freund
Hi, On 2023-02-11 14:25:06 -0800, Andres Freund wrote: > On 2023-01-20 13:40:55 +1300, David Rowley wrote: > > v2-0004: > > > > 5. Is it worth having two versions of PinLocalBuffer() one to adjust > > the usage count and one that does not? Couldn't the version that does > > not adjust the count s

Re: refactoring relation extension and BufferAlloc(), faster COPY

2023-02-11 Thread Andres Freund
Hi, On 2023-01-20 13:40:55 +1300, David Rowley wrote: > On Tue, 10 Jan 2023 at 15:08, Andres Freund wrote: > > Thanks for letting me now. Updated version attached. > > I'm not too sure I've qualified for giving a meaningful design review > here, but I have started looking at the patches and so f

Re: refactoring relation extension and BufferAlloc(), faster COPY

2023-02-11 Thread Andres Freund
Hi, On 2023-02-10 18:38:50 +0200, Heikki Linnakangas wrote: > I'll continue reviewing this, but here's some feedback on the first two > patches: > > v2-0001-aio-Add-some-error-checking-around-pinning.patch: > > I wonder if the extra assertion in LockBufHdr() is worth the overhead. It > won't add

Re: refactoring relation extension and BufferAlloc(), faster COPY

2023-02-11 Thread Andres Freund
On 2023-02-11 13:36:51 -0800, Andres Freund wrote: > Even though it's not a correctness issue, it seems to me that > DropRelationsAllBuffers() etc ought to check if the buffer is BM_TAG_VALID, > before doing anything further. Particularly in DropRelationsAllBuffers(), the > check we do for each bu

Re: Transparent column encryption

2023-02-11 Thread Mark Dilger
> On Jan 25, 2023, at 10:44 AM, Peter Eisentraut > wrote: > > Here is a new patch. Changes since v14: > > - Fixed some typos (review by Justin Pryzby) > - Fixed backward compat. psql and pg_dump (review by Justin Pryzby) > - Doc additions (review by Jacob Champion) > - Validate column_encry

Re: Improving inferred query column names

2023-02-11 Thread Corey Huinker
On Sat, Feb 11, 2023 at 3:47 PM Vladimir Churyukin wrote: > For backwards compatibility I guess you can have a GUC flag controlling > that behavior that can be set into backwards compatibility mode if required. > The previous functionality can be declared deprecated and removed (with > the flag)

Re: refactoring relation extension and BufferAlloc(), faster COPY

2023-02-11 Thread Andres Freund
Hi, On 2023-02-11 23:03:56 +0200, Heikki Linnakangas wrote: > > v2-0005-bufmgr-Acquire-and-clean-victim-buffer-separately.patch > This can be applied separately from the rest of the patches, which is nice. > Some small comments on it: Thanks for looking at these! > * Needs a rebase, it conflict

Re: proposal: psql: psql variable BACKEND_PID

2023-02-11 Thread Andres Freund
On 2023-02-04 15:35:58 -0500, Corey Huinker wrote: > This effectively makes the %p prompt (which I use in the example above) the > same as %:BACKEND_PID: and we may want to note that in the documentation. I don't really see much of a point in noting this in the doc. I don't know in what situation

Re: refactoring relation extension and BufferAlloc(), faster COPY

2023-02-11 Thread Heikki Linnakangas
v2-0005-bufmgr-Acquire-and-clean-victim-buffer-separately.patch This can be applied separately from the rest of the patches, which is nice. Some small comments on it: * Needs a rebase, it conflicted slightly with commit f30d62c2fc. * GetVictimBuffer needs a comment to explain what it does. In

Re: proposal: psql: psql variable BACKEND_PID

2023-02-11 Thread Andres Freund
Hi, On 2023-02-09 10:11:21 +0100, Pavel Stehule wrote: > first and main (for me) - I can use psql variables tab complete - just > :B - it is significantly faster > second - I can see all connection related information by \set > third - there is not hook on reconnect in psql - so if you implement >

Re: Improving inferred query column names

2023-02-11 Thread Vladimir Churyukin
That is a good idea for simple cases, I'm just curious how it would look like for more complex cases (you can have all kinds of expressions as parameters for aggregate function calls). If it works only for simple cases, I think it would be confusing and not very helpful. Wouldn't it make more sense

Re: Sort optimizations: Making in-memory sort cache-aware

2023-02-11 Thread Andres Freund
Hi, On 2023-02-11 17:49:02 +0530, Ankit Kumar Pandey wrote: > 2. Frequent cache misses > > Issue #1 is being looked in separate patch. I am currently looking at #2. > > Possible solution was to batch tuples into groups (which can fit into L3 > cache) before pushing them to sort function. > > After

Improving inferred query column names

2023-02-11 Thread Andres Freund
Hi, A common annoyance when writing ad-hoc analytics queries is column naming once aggregates are used. Useful column names: SELECT reads, writes FROM pg_stat_io; column names: reads, writes Not useful column names: SELECT SUM(reads), SUM(writes) FROM pg_stat_io; column names: sum, sum So i oft

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-02-11 Thread Andres Freund
Hi, On 2023-02-08 21:03:19 -0800, Andres Freund wrote: > Pushed the first (and biggest) commit. More tomorrow. Just pushed the actual pg_stat_io view, the splitting of the tablespace test, and the pg_stat_io tests. Yay! Thanks all for patch and review! > Already can't wait to see incremental

Re: UUID v7

2023-02-11 Thread Peter Eisentraut
On 11.02.23 02:14, Andres Freund wrote: On 2023-02-10 15:57:50 -0800, Andrey Borodin wrote: As you may know there's a new version of UUID being standardized [0]. These new algorithms of UUID generation are very promising for database performance. I agree it's very useful to have. [0] https

Sort optimizations: Making in-memory sort cache-aware

2023-02-11 Thread Ankit Kumar Pandey
Hi all, While working on sort optimization for window function, it was seen that performance of sort where all tuples are in memory was bad when number of tuples were very large [1] Eg: work_mem = 4 GB, sort on 4 int columns on table having 10 million tuples. Issues we saw were as follows

Re: pg_stat_statements and "IN" conditions

2023-02-11 Thread Dmitry Dolgov
> On Sat, Feb 11, 2023 at 11:47:07AM +0100, Dmitry Dolgov wrote: > > The original version of the patch was doing all of this, i.e. handling > numerics, Param nodes, RTE_VALUES. The commentary about > find_const_walker in tests is referring to a part of that, that was > dealing with evaluation of ex

Re: pg_stat_statements and "IN" conditions

2023-02-11 Thread Dmitry Dolgov
> On Sat, Feb 11, 2023 at 11:03:36AM +0100, David Geier wrote: > Hi, > > On 2/9/23 16:02, Dmitry Dolgov wrote: > > > Unfortunately, rebase is needed again due to recent changes in > > > queryjumblefuncs ( 9ba37b2cb6a174b37fc51d0649ef73e56eae27fc ) > I reviewed the last patch applied to some commit

Re: pg_stat_statements and "IN" conditions

2023-02-11 Thread David Geier
Hi, On 2/9/23 16:02, Dmitry Dolgov wrote: Unfortunately, rebase is needed again due to recent changes in queryjumblefuncs ( 9ba37b2cb6a174b37fc51d0649ef73e56eae27fc ) I reviewed the last patch applied to some commit from Feb. 4th. It seems a little strange to me that with const_merge_threshol