Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

2025-01-18 Thread Sami Imseih
> Hmm. I am reading Tom's opinion that goes toward not going in this > direction for more commands, with the point to extend EXPLAIN to show > this kind of information: > https://www.postgresql.org/message-id/1692530.1736369...@sss.pgh.pa.us That sounds like the ability to do something like EXPLA

Re: Re:Limit length of queryies in pg_stat_statement extension

2025-01-18 Thread Julien Rouhaud
On Sun, Jan 19, 2025 at 03:32:19AM +0800, 赵庭海(庭章) wrote: > > I'm sorry I left out some details earlier. I found that the garbage collect > backend process was in the loop of gc_qtexts while for a long time. The main > backtrace is below. > > ``` > #0 0x7fc528d6aba0 in __write_nocancel () from

Re: Coccinelle for PostgreSQL development [1/N]: coccicheck.py

2025-01-18 Thread Michael Paquier
On Sat, Jan 18, 2025 at 08:44:00PM +0100, Mats Kindahl wrote: > For PostgreSQL 16, Peter extended the palloc()/pg_malloc() interface in > commit 2016055a92f to provide more type-safety, but these functions are not > widely used. This semantic patch captures and replaces all uses of palloc() > where

Re: Pgoutput not capturing the generated columns

2025-01-18 Thread Peter Smith
Hi Vignesh, I was having some second thoughts about this patch and my previous suggestion. Currently the code is current written something like: printfPQExpBuffer(&buf, "SELECT oid, pubname,\n" " pg_catalog.pg_get_userbyid(pubowner) AS owner,\n" " puballtables, pubinsert, pubupdate, pubdele

Re: Issue with markers in isolation tester? Or not?

2025-01-18 Thread Michael Paquier
On Sat, Jan 18, 2025 at 02:03:33PM +0100, Michail Nikolaev wrote: > I was able to stabilize (I hope so) tests on [0] without any changes to > isolationtester and without "notices N". > > So, I agree, it is better to put it on a shelf now. But a few words in > documentation may be a good idea. Cou

Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

2025-01-18 Thread Michael Paquier
On Sat, Jan 18, 2025 at 05:42:50PM -0600, Sami Imseih wrote: > Here is a v1 that implements CREATE INDEX (VERBOSE) > to output the parallel workers planned and launched. Hmm. I am reading Tom's opinion that goes toward not going in this direction for more commands, with the point to extend EXPLAI

Re: Statistics Import and Export

2025-01-18 Thread Michael Paquier
On Sun, Jan 19, 2025 at 01:00:04AM +0800, jian he wrote: > On Fri, Jan 17, 2025 at 10:20 PM jian he wrote: >> dump and execute the above query generated a warning >> WARNING: missing lock for relation "tenk1_hundred" (OID 18431, >> relkind i) @ TID (15,34) > > This seems to be an existing issue.

Re: Old BufferDesc refcount in PrintBufferDescs and PrintPinnedBufs

2025-01-18 Thread Michael Paquier
On Fri, Jan 17, 2025 at 11:14:02PM -0800, Jacob Brazeal wrote: > Sounds good! v3 attached. Removal sounds good to me. Any objections from anybody? Andres, perhaps you have some experience using that and would prefer keep them and make them work? -- Michael signature.asc Description: PGP signat

Re: Get rid of WALBufMappingLock

2025-01-18 Thread Yura Sokolov
19.01.2025 03:11, Yura Sokolov пишет: Good day, hackers. During discussion of Increasing NUM_XLOGINSERT_LOCKS [1], Andres Freund used benchmark which creates WAL records very intensively. While I this it is not completely fair (1MB log records are really rare), it pushed me to analyze write-s

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-18 Thread Yura Sokolov
17.01.2025 17:00, Zhou, Zhiguo пишет: On 1/16/2025 10:00 PM, Yura Sokolov wrote: Good day, Zhiguo. Excuse me, I feel sneaky a bit, but I've started another thread just about increase of NUM_XLOGINSERT_LOCK, because I can measure its effect even on my working notebook (it is another one: Ry

Get rid of WALBufMappingLock

2025-01-18 Thread Yura Sokolov
Good day, hackers. During discussion of Increasing NUM_XLOGINSERT_LOCKS [1], Andres Freund used benchmark which creates WAL records very intensively. While I this it is not completely fair (1MB log records are really rare), it pushed me to analyze write-side waiting of XLog machinery. First

Re: improve DEBUG1 logging of parallel workers for CREATE INDEX?

2025-01-18 Thread Sami Imseih
The opinion in this thread is leaning towards a VERBOSE option and I agree with this as it provides the users with a much more intuitive way to gather parallel information about parallel index builds. Also, the current DEBUG1 output is useless as it is now. Here is a v1 that implements CREATE INDE

Re: [PATCH] immediately kill psql process if server is not running.

2025-01-18 Thread Tom Lane
I wrote: > Anyway, I'm inclined to close this CF entry as returned-with-feedback. > Perhaps there's something we can do here, but the current patch feels > like a dead end. Also, the submitter seems to have disappeared: - The following addresses had permanent fatal errors - (reas

Re: [PATCH] immediately kill psql process if server is not running.

2025-01-18 Thread Tom Lane
Tomas Vondra writes: > Yes, it's true that if the connection breaks, the client may not notice > that until the next command. But why would we want to kill psql in that > case? How does that improve the user experience? > If you're in an interactive session (which is where this matters), and > yo

Re: Confine vacuum skip logic to lazy_scan_skip

2025-01-18 Thread Tomas Vondra
On 1/18/25 22:31, Thomas Munro wrote: > On Sun, Jan 19, 2025 at 5:51 AM Tomas Vondra wrote: >> * Does it still make sense to default to eic=1? For this particular test >> increasing eic=4 often cuts the duration in half (especially on nvme >> storage). > > Maybe it wasn't a bad choice for systems

Re: Confine vacuum skip logic to lazy_scan_skip

2025-01-18 Thread Thomas Munro
On Sun, Jan 19, 2025 at 10:31 AM Thomas Munro wrote: > read_stream.c doesn't do that > sort of multiplication itself, Actually for completeness there is a place where it allocates local memory for max I/Os * 4, and that 4 is a not entirely unbogus and should change to io_combine_limit for the AIO

Re: [PATCH] Fix jsonb comparison for raw scalar pseudo arrays

2025-01-18 Thread Andrew Dunstan
On 2025-01-18 Sa 1:17 PM, Tom Lane wrote: Andrew Dunstan writes: Pushed something along these lines. In master I also added a code comment so nobody might be tempted to "fix" the anomaly. There is still a commitfest entry [1] pointing at this thread. Should it be closed as committed, or is t

Re: Confine vacuum skip logic to lazy_scan_skip

2025-01-18 Thread Thomas Munro
On Sun, Jan 19, 2025 at 5:51 AM Tomas Vondra wrote: > * Does it still make sense to default to eic=1? For this particular test > increasing eic=4 often cuts the duration in half (especially on nvme > storage). Maybe it wasn't a bad choice for systems with one spinning disk, but obviously typical

Re: InitControlFile misbehaving on graviton

2025-01-18 Thread Julian Andres Klode
On Mon, Jan 13, 2025 at 09:39:40PM +0100, Matthias van de Meent wrote: > On Mon, 13 Jan 2025 at 20:04, Christoph Berg wrote: > > > > Bernd and I have been chasing a bug that happens when all of the > > following conditions are fulfilled: > > > > * PG 15..18 (older PGs are ok) > > * gcc 14.2 on Deb

Re: Adding comments to help understand psql hidden queries

2025-01-18 Thread Tom Lane
David Christensen writes: > Any further concerns/issues with this patch that I can address to help > move it forward? I got around to looking at this finally --- sorry that it's been on the back burner for so long. I think this is basically a good idea but it still requires a lot of sanding-down

Re: [PATCH] Fix jsonb comparison for raw scalar pseudo arrays

2025-01-18 Thread Tom Lane
Andrew Dunstan writes: > Pushed something along these lines. In master I also added a code > comment so nobody might be tempted to "fix" the anomaly. There is still a commitfest entry [1] pointing at this thread. Should it be closed as committed, or is there more to do?

Re: Replace current implementations in crypt() and gen_salt() to OpenSSL

2025-01-18 Thread Joe Conway
On 1/15/25 09:24, Daniel Gustafsson wrote: On 14 Jan 2025, at 13:12, Hayato Kuroda (Fujitsu) wrote: Similar with [1], `pg_gen_salt_rounds` is not an SQL function. I think we do not have to mention the function because it's just another implementation of gen_salt(). Also, use instead of .

Re: abi-compliance-checker

2025-01-18 Thread David E. Wheeler
Greetings old thread. On Mar 4, 2024, at 07:50, Peter Eisentraut wrote: > Maybe the way forward here is to write a buildfarm module for this, and then > see from there what further needs there are. Given the PostgreSQL 17.1 kerfuffle and the addition of API and API guidance to the docs (discu

Re: Timeline issue if StartupXLOG() is interrupted right before end-of-recovery record is done

2025-01-18 Thread Andrey M. Borodin
Hi Roman! Thanks for raising the issue. I think the root cause is that many systems imply that higher number of timeline id means more recent timeline write. This invariant is not uphold. It's not even more recent timeline start. "latest timeline" effectively means "random timeline". > On 17 Jan

Re: Statistics Import and Export

2025-01-18 Thread jian he
On Fri, Jan 17, 2025 at 10:20 PM jian he wrote: > > hi. > > SELECT * FROM pg_catalog.pg_restore_relation_stats( > 'relation', 'public.tenk1_hundred'::regclass, > 'version', '18'::integer, > 'relpages', '11'::integer, > 'reltuples', '1'::real, > 'relallvisible', '0'::int

Re: Eagerly scan all-visible pages to amortize aggressive vacuum

2025-01-18 Thread Robert Treat
On Wed, Jan 15, 2025 at 3:56 PM Melanie Plageman wrote: > On Wed, Jan 15, 2025 at 3:03 PM Melanie Plageman > wrote: > > > > On Wed, Jan 15, 2025 at 2:36 PM Marcos Pegoraro wrote: > > >> > > > There is a typo on committed vacuumlazy.c file > > > * If the TID store fills up in phase I, vacuum sus

Re: Issue with markers in isolation tester? Or not?

2025-01-18 Thread Michail Nikolaev
Hello, everyone! I was able to stabilize (I hope so) tests on [0] without any changes to isolationtester and without "notices N". So, I agree, it is better to put it on a shelf now. But a few words in documentation may be a good idea. Best regards, Mikhail. [0]: https://www.postgresql.org/messa

Re: Issues with ON CONFLICT UPDATE and REINDEX CONCURRENTLY

2025-01-18 Thread Michail Nikolaev
Hello, everyone! This is just small test refactoring after the last stabilization. Best regards, Mikhail. > v7-0001-Specs-top-reproduce-the-issues-with-CREATE-INDEX.patch Description: Binary data v7-0004-Modify-the-ExecInitPartitionInfo-function-to-cons.patch Description: Binary data v7-00

Re: Increase NUM_XLOGINSERT_LOCKS

2025-01-18 Thread Yura Sokolov
Since it seems Andres missed my request to send answer's copy, here it is: On 2025-01-16 18:55:47 +0300, Yura Sokolov wrote: > 16.01.2025 18:36, Andres Freund пишет: >> Hi, >> >> On 2025-01-16 16:52:46 +0300, Yura Sokolov wrote: >>> Good day, hackers. >>> >>> Zhiguo Zhow proposed to transform xlo

Re: Parallel heap vacuum

2025-01-18 Thread Dilip Kumar
On Fri, Jan 17, 2025 at 10:43 PM Masahiko Sawada wrote: > On Fri, Jan 17, 2025 at 1:43 AM Dilip Kumar wrote: > > > > On Fri, Jan 17, 2025 at 6:37 AM Masahiko Sawada > wrote: > >> > >> On Sun, Jan 12, 2025 at 1:34 AM Masahiko Sawada > wrote: > >> > > > > > > > IIRC, there was one of the blocker

Re: jsonb_strip_nulls with arrays?

2025-01-18 Thread Florents Tselai
> On 8 Jan 2025, at 6:45 PM, Andrew Dunstan wrote: > > > > On 2024-09-17 Tu 4:53 PM, Florents Tselai wrote: >> >>> We could, if we're going to do anything at all in this area. Another >>> possibility would be to provide a second optional parameter for >>> json{b}_strip_nulls. That's probab

Re: create subscription with (origin = none, copy_data = on)

2025-01-18 Thread Sergey Tatarintsev
18.01.2025 12:01, vignesh C пишет: On Fri, 17 Jan 2025 at 21:30, vignesh C wrote: On Fri, 17 Jan 2025 at 14:00, Sergey Tatarintsev wrote: Hi, hackers! I am looking at subscription creation command: CREATE SUBSCRIPTION sub CONNECTION '...' PUBLICATION pub WITH (origin = none, copy_data = on)

Re: per backend I/O statistics

2025-01-18 Thread Michael Paquier
On Fri, Jan 17, 2025 at 10:23:48AM +, Bertrand Drouvot wrote: > Please find attached a patch implementing the ideas above, meaning: > > - It creates a new PendingBackendStats variable > - It uses this variable to increment and flush per backend pending IO > statistics > > That way we get rid