Re: Reducing the log spam

2025-04-05 Thread Tom Lane
Fujii Masao writes: > Filtering log messages by SQLSTATE might be useful for some users, > but I'm unsure if it should belong in the core. There are also other > potential filtering needs, such as by application name, client host, > database, or roles. Adding only SQLSTATE filtering may not be goo

Re: Reducing the log spam

2025-04-05 Thread Fujii Masao
On 2025/03/17 17:12, Jim Jones wrote: On 15.03.25 07:12, Laurenz Albe wrote: ... and here is v7, improved in the spirit of https://postgr.es/m/E132D362-A669-4606-AFE1-B45C9DFCC141%40yesql.se I just revisited this patch and v7 passes all tests from [1,2]. LGTM. Since this patch is marked

Re: Reducing the log spam

2025-04-03 Thread Laurenz Albe
On Wed, 2025-04-02 at 15:23 -0400, Tom Lane wrote: > Fujii Masao writes: > > Filtering log messages by SQLSTATE might be useful for some users, > > but I'm unsure if it should belong in the core. There are also other > > potential filtering needs, such as by application name, client host, > > data

Re: Reducing the log spam

2025-03-17 Thread Jim Jones
On 15.03.25 07:12, Laurenz Albe wrote: > ... and here is v7, improved in the spirit of > https://postgr.es/m/E132D362-A669-4606-AFE1-B45C9DFCC141%40yesql.se I just revisited this patch and v7 passes all tests from [1,2]. LGTM. Best regards, Jim [1] - https://www.postgresql.org/message-id/2055

Re: Reducing the log spam

2025-03-14 Thread Laurenz Albe
On Fri, 2025-03-14 at 06:58 +0100, Laurenz Albe wrote: > On Tue, 2025-03-11 at 10:46 +0100, Laurenz Albe wrote: > > Attached is the fifth version of the patch. > > ... and here, for good measure, a pgindented version > Apart from that, no changes. ... and here is v7, improved in the spirit of htt

Re: Reducing the log spam

2025-03-13 Thread Laurenz Albe
On Tue, 2025-03-11 at 10:46 +0100, Laurenz Albe wrote: > Attached is the fifth version of the patch. ... and here, for good measure, a pgindented version Apart from that, no changes. Yours, Laurenz Albe From d644d3fe1b2c5ab2cbf7f10e9c653b7baa9efe01 Mon Sep 17 00:00:00 2001 From: Laurenz Albe Dat

Re: Reducing the log spam

2025-03-11 Thread Laurenz Albe
On Fri, 2025-03-07 at 20:38 +0100, Jim Jones wrote: > I've tested this patch and for the most part it works as intended. Thanks for the thorough test! > There are a few issues though ... > > 1) Invalid codes aren't rejected. Is there any way to validate them? > > postgres=# SET log_suppress_err

Re: Reducing the log spam

2025-03-11 Thread Jim Jones
On 11.03.25 10:46, Laurenz Albe wrote: > Thanks for the thorough test! > >> There are a few issues though ... >> >> 1) Invalid codes aren't rejected. Is there any way to validate them? >> >> postgres=# SET log_suppress_errcodes TO '0foo1'; SHOW log_suppress_errcodes; >> SET >>  log_suppress_errco

Re: Reducing the log spam

2025-03-07 Thread Jim Jones
Hi Laurenz On 18.06.24 18:49, Laurenz Albe wrote: > I have attached a new version that leaves the parameter empty by default. I've tested this patch and for the most part it works as intended. For convenience, I wrote a small function to simulate the exceptions using a given errcode: CREATE OR

Re: Reducing the log spam

2024-08-15 Thread Rafia Sabih
On Thu, 25 Jul 2024 at 18:03, Laurenz Albe wrote: > Thanks for the review! > > On Wed, 2024-07-24 at 15:27 +0200, Rafia Sabih wrote: > > I liked the idea for this patch. I will also go for the default being > > an empty string. > > I went through this patch and have some comments on the code, > >

Re: Reducing the log spam

2024-07-25 Thread Laurenz Albe
Thanks for the review! On Wed, 2024-07-24 at 15:27 +0200, Rafia Sabih wrote: > I liked the idea for this patch. I will also go for the default being > an empty string. > I went through this patch and have some comments on the code, > > 1. In general, I don't like the idea of goto, maybe we can ha

Re: Reducing the log spam

2024-07-24 Thread Rafia Sabih
Hello Laurenz, I liked the idea for this patch. I will also go for the default being an empty string. I went through this patch and have some comments on the code, 1. In general, I don't like the idea of goto, maybe we can have a free_something function to call here. 2. if (!SplitIdentifierStrin

Re: Reducing the log spam

2024-06-18 Thread Laurenz Albe
On Mon, 2024-06-17 at 16:40 -0500, Justin Pryzby wrote: > > The feature will become much less useful if unique voilations keep getting > > logged. > > Uh, to be clear, your patch is changing the *defaults*, which I found > surprising, even after reaading the thread. Evidently, the current > beha

Re: Reducing the log spam

2024-06-17 Thread Justin Pryzby
On Thu, May 02, 2024 at 12:47:45PM +0200, Laurenz Albe wrote: > On Mon, 2024-03-11 at 09:33 +0100, Jelte Fennema-Nio wrote: > > -   the subscriber's server log. > > +   the subscriber's server log if you remove 23505 from > > +   . > > > > This seems like a pretty big regression. Being able to kno

Re: Reducing the log spam

2024-05-03 Thread Laurenz Albe
On Thu, 2024-05-02 at 13:11 +0200, Jelte Fennema-Nio wrote: > On Thu, 2 May 2024 at 13:08, Jelte Fennema-Nio wrote: > > 2. Change the newly added check in errcode() to only set > > output_to_server to false when IsLogicalWorker() returns false. > > Actually a third, and probably even better solut

Re: Reducing the log spam

2024-05-02 Thread Jelte Fennema-Nio
On Thu, 2 May 2024 at 13:08, Jelte Fennema-Nio wrote: > 2. Change the newly added check in errcode() to only set > output_to_server to false when IsLogicalWorker() returns false. Actually a third, and probably even better solution would be to only apply this new GUC to non-backgroundworker proces

Re: Reducing the log spam

2024-05-02 Thread Jelte Fennema-Nio
On Thu, 2 May 2024 at 12:47, Laurenz Albe wrote: > Yes. But I'd argue that that is a shortcoming of logical replication: > there should be a ways to get this information via SQL. Having to look into > the log file is not a very useful option. Definitely agreed that accessing the error details u

Re: Reducing the log spam

2024-05-02 Thread Laurenz Albe
On Mon, 2024-03-11 at 09:33 +0100, Jelte Fennema-Nio wrote: > -   the subscriber's server log. > +   the subscriber's server log if you remove 23505 from > +   . > > This seems like a pretty big regression. Being able to know why your > replication got closed seems pretty critical. Yes. But I'd

Re: Reducing the log spam

2024-03-11 Thread Laurenz Albe
On Mon, 2024-03-11 at 09:33 +0100, Jelte Fennema-Nio wrote: > -   the subscriber's server log. > +   the subscriber's server log if you remove 23505 from > +   . > > This seems like a pretty big regression. Being able to know why your > replication got closed seems pretty critical. The actual SQL

Re: Reducing the log spam

2024-03-11 Thread Jelte Fennema-Nio
- the subscriber's server log. + the subscriber's server log if you remove 23505 from + . This seems like a pretty big regression. Being able to know why your replication got closed seems pretty critical. On Mon, 11 Mar 2024 at 03:44, Laurenz Albe wrote: > > On Sat, 2024-03-09 at 14:03 +01

Re: Reducing the log spam

2024-03-10 Thread Laurenz Albe
On Sat, 2024-03-09 at 14:03 +0100, Laurenz Albe wrote: > Here is a patch that implements this. And here is patch v2 that fixes a bug and passes the regression tests. Yours, Laurenz Albe From 6c72ea7d0aa1df569a4e53f54fcb1a11542ac0ef Mon Sep 17 00:00:00 2001 From: Laurenz Albe Date: Mon, 11 Mar 20

Re: Reducing the log spam

2024-03-09 Thread Laurenz Albe
On Thu, 2024-03-07 at 08:30 +0100, Laurenz Albe wrote: > On Wed, 2024-03-06 at 17:33 -0500, Isaac Morland wrote: > > I have two questions about this: > > > > First, can it be done per role? If I have a particular application which is > > constantly throwing some particular error, I might want to s

Re: Reducing the log spam

2024-03-06 Thread Laurenz Albe
On Wed, 2024-03-06 at 17:33 -0500, Isaac Morland wrote: > I have two questions about this: > > First, can it be done per role? If I have a particular application which is > constantly throwing some particular error, I might want to suppress it, but > not suppress the same error occasionally coming

Re: Reducing the log spam

2024-03-06 Thread Isaac Morland
On Tue, 5 Mar 2024 at 07:55, Laurenz Albe wrote: > Inspired by feedback to [1], I thought about how to reduce log spam. > > My experience from the field is that a lot of log spam looks like > > database/table/... "xy" does not exist > duplicate key value violates unique constraint "xy" > > So

Re: Reducing the log spam

2024-03-06 Thread Laurenz Albe
On Wed, 2024-03-06 at 10:50 -0500, Greg Sabino Mullane wrote: > On Tue, Mar 5, 2024 at 7:55 AM Laurenz Albe wrote: > > My experience from the field is that a lot of log spam looks like > > > >   database/table/... "xy" does not exist > >   duplicate key value violates unique constraint "xy" > >

Re: Reducing the log spam

2024-03-06 Thread Laurenz Albe
On Wed, 2024-03-06 at 17:09 +0300, Aleksander Alekseev wrote: > I like the idea of suppressing certain log messages in general, but > the particular user interface doesn't strike me as an especially > convenient one. > > Firstly I don't think many people remember sqlstates and what 3F000 > stands

Re: Reducing the log spam

2024-03-06 Thread Greg Sabino Mullane
On Tue, Mar 5, 2024 at 7:55 AM Laurenz Albe wrote: > My experience from the field is that a lot of log spam looks like > > database/table/... "xy" does not exist > duplicate key value violates unique constraint "xy" Forcibly hiding those at the Postgres level seems a heavy hammer for what i

Re: Reducing the log spam

2024-03-06 Thread Aleksander Alekseev
Hi, > So what about a parameter "log_suppress_sqlstates" that suppresses > logging ERROR and FATAL messages with the enumerated SQL states? > > My idea for a default setting would be something like > > log_suppress_sqlstates = '23505,3D000,3F000,42601,42704,42883,42P01' > > but that's of course

Re: Reducing the log spam

2024-03-05 Thread Jelte Fennema-Nio
On Tue, 5 Mar 2024 at 14:55, Jim Jones wrote: > > So what about a parameter "log_suppress_sqlstates" that suppresses > > logging ERROR and FATAL messages with the enumerated SQL states? Big +1 from me for this idea.

Re: Reducing the log spam

2024-03-05 Thread Pavel Stehule
Hi út 5. 3. 2024 v 14:55 odesílatel Jim Jones napsal: > Hi Laurenz > > On 05.03.24 13:55, Laurenz Albe wrote: > > Inspired by feedback to [1], I thought about how to reduce log spam. > > > > My experience from the field is that a lot of log spam looks like > > > >database/table/... "xy" does

Re: Reducing the log spam

2024-03-05 Thread Jim Jones
Hi Laurenz On 05.03.24 13:55, Laurenz Albe wrote: Inspired by feedback to [1], I thought about how to reduce log spam. My experience from the field is that a lot of log spam looks like database/table/... "xy" does not exist duplicate key value violates unique constraint "xy" So what abo

Re: Reducing the log spam

2024-03-05 Thread Pavel Stehule
Hi út 5. 3. 2024 v 13:55 odesílatel Laurenz Albe napsal: > Inspired by feedback to [1], I thought about how to reduce log spam. > > My experience from the field is that a lot of log spam looks like > > database/table/... "xy" does not exist > duplicate key value violates unique constraint "x

Reducing the log spam

2024-03-05 Thread Laurenz Albe
Inspired by feedback to [1], I thought about how to reduce log spam. My experience from the field is that a lot of log spam looks like database/table/... "xy" does not exist duplicate key value violates unique constraint "xy" So what about a parameter "log_suppress_sqlstates" that suppresses