Re: BackgroundPsql swallowing errors on windows

2025-03-11 Thread Andres Freund
Hi, On 2025-02-16 17:52:36 -0800, Noah Misch wrote: > On Sun, Feb 16, 2025 at 08:42:50PM -0500, Andres Freund wrote: > > On February 16, 2025 7:50:18 PM EST, Tom Lane wrote: > > >Noah Misch writes: > > >> On Sun, Feb 16, 2025 at 06:18:44PM -0500, Tom Lane wrote: > > >>> I think that > > >>> IPC:

Re: BackgroundPsql swallowing errors on windows

2025-03-09 Thread Noah Misch
On Sun, Mar 09, 2025 at 12:47:34PM -0400, Andres Freund wrote: > On 2025-02-16 17:52:36 -0800, Noah Misch wrote: > > On Sun, Feb 16, 2025 at 08:42:50PM -0500, Andres Freund wrote: > > > On February 16, 2025 7:50:18 PM EST, Tom Lane wrote: > > > >Noah Misch writes: > > > >> On Sun, Feb 16, 2025 at

Re: BackgroundPsql swallowing errors on windows

2025-02-19 Thread Andres Freund
Hi, On 2025-02-19 10:41:53 -0500, Andres Freund wrote: > I'm planning to wait for a few hours and see what the BF says and then push > the fix this thread is about to all supported branches. Not having seen any issues, I pushed the fix. Thanks for the reviews etc! Greetings, Andres Freund

Re: BackgroundPsql swallowing errors on windows

2025-02-19 Thread Andres Freund
Hi, On 2025-02-17 09:24:24 +0900, Michael Paquier wrote: > On Sun, Feb 16, 2025 at 07:03:39PM -0500, Andres Freund wrote: > > Cool, will after the minor release freeze. > > Thanks, Andres. If you'd prefer that I double-check the code and do > it as the former committer of these two ones, please l

Re: BackgroundPsql swallowing errors on windows

2025-02-16 Thread Thomas Munro
On Mon, Feb 17, 2025 at 7:02 AM Andres Freund wrote: > I don't really know enough about IPC::Run's internals to answer. My > interpretation of how it might work, purely from observation, is that it opens > one tcp connection for each "pipe" and that that's what's introducing the > potential of reo

Re: BackgroundPsql swallowing errors on windows

2025-02-16 Thread Tom Lane
Noah Misch writes: > On Sun, Feb 16, 2025 at 08:42:50PM -0500, Andres Freund wrote: >> On macos we can't currently use images, so we just cache all the installed >> macports packages. The cache is keyed by OS version and list of packages to >> be installed, with no other forced invalidation righ

Re: BackgroundPsql swallowing errors on windows

2025-02-16 Thread Andres Freund
Hi, On February 16, 2025 8:42:50 PM EST, Andres Freund wrote: >On Windows cpan is used, so it should pick that new version fairly quickly if >a release has been made. Looks like no release has happened since late 2023: https://github.com/cpan-authors/IPC-Run/blob/master/Changelog So it won't

Re: BackgroundPsql swallowing errors on windows

2025-02-16 Thread Noah Misch
On Sun, Feb 16, 2025 at 08:42:50PM -0500, Andres Freund wrote: > On February 16, 2025 7:50:18 PM EST, Tom Lane wrote: > >Noah Misch writes: > >> On Sun, Feb 16, 2025 at 06:18:44PM -0500, Tom Lane wrote: > >>> I think that > >>> IPC::Run may be screwing up here, because I have seen non-Windows > >

Re: BackgroundPsql swallowing errors on windows

2025-02-16 Thread Andres Freund
Hi, On February 16, 2025 7:50:18 PM EST, Tom Lane wrote: >Noah Misch writes: >> On Sun, Feb 16, 2025 at 06:18:44PM -0500, Tom Lane wrote: >>> I think that >>> IPC::Run may be screwing up here, because I have seen non-Windows >>> CI failures that look like it didn't read all the stderr output. >

Re: BackgroundPsql swallowing errors on windows

2025-02-16 Thread Tom Lane
Noah Misch writes: > On Sun, Feb 16, 2025 at 06:18:44PM -0500, Tom Lane wrote: >> I think that >> IPC::Run may be screwing up here, because I have seen non-Windows >> CI failures that look like it didn't read all the stderr output. >> For example, this pgbench test failure on macOS from [1]: > ht

Re: BackgroundPsql swallowing errors on windows

2025-02-16 Thread Michael Paquier
On Sun, Feb 16, 2025 at 07:03:39PM -0500, Andres Freund wrote: > git {show|log|...} --abbrev-commit Ah, thanks for the hint. This can also be controlled by core.abbrev at repo level with %h. Didn't suspect this keyword. > Cool, will after the minor release freeze. Thanks, Andres. If you'd pre

Re: BackgroundPsql swallowing errors on windows

2025-02-16 Thread Andres Freund
Hi, On 2025-02-17 08:52:58 +0900, Michael Paquier wrote: > On Sat, Feb 15, 2025 at 04:13:37PM -0500, Andres Freund wrote: > > On 2025-02-14 09:52:24 -0800, Jacob Champion wrote: > >> On Fri, Feb 14, 2025 at 8:53 AM Andres Freund wrote: > >>> commit 70291a3c66e > > (Side question entirely unrelat

Re: BackgroundPsql swallowing errors on windows

2025-02-16 Thread Andres Freund
Hi, On 2025-02-16 18:18:44 -0500, Tom Lane wrote: > Noah Misch writes: > > From the slow proxy's perspective, it can't rule out the program under test > > having done those two write() calls. The proxy doesn't have enough > > information to reconstruct the original four write() calls. What prev

Re: BackgroundPsql swallowing errors on windows

2025-02-16 Thread Noah Misch
On Sun, Feb 16, 2025 at 06:18:44PM -0500, Tom Lane wrote: > Noah Misch writes: > > From the slow proxy's perspective, it can't rule out the program under test > > having done those two write() calls. The proxy doesn't have enough > > information to reconstruct the original four write() calls. Wh

Re: BackgroundPsql swallowing errors on windows

2025-02-16 Thread Michael Paquier
On Sat, Feb 15, 2025 at 04:13:37PM -0500, Andres Freund wrote: > On 2025-02-14 09:52:24 -0800, Jacob Champion wrote: >> On Fri, Feb 14, 2025 at 8:53 AM Andres Freund wrote: >>> commit 70291a3c66e (Side question entirely unrelated as I'm reading that..) What's your magic recipe for showing up with

Re: BackgroundPsql swallowing errors on windows

2025-02-16 Thread Tom Lane
Noah Misch writes: > From the slow proxy's perspective, it can't rule out the program under test > having done those two write() calls. The proxy doesn't have enough > information to reconstruct the original four write() calls. What prevents > that anomaly? Yeah, I think it's hopeless to expect

Re: BackgroundPsql swallowing errors on windows

2025-02-16 Thread Noah Misch
On Sun, Feb 16, 2025 at 03:55:10PM -0500, Andres Freund wrote: > On 2025-02-16 10:47:40 -0800, Noah Misch wrote: > > On Sun, Feb 16, 2025 at 01:02:01PM -0500, Andres Freund wrote: > > > On 2025-02-16 09:39:43 -0800, Noah Misch wrote: > > > > On Thu, Feb 13, 2025 at 12:39:04PM -0500, Andres Freund w

Re: BackgroundPsql swallowing errors on windows

2025-02-16 Thread Andres Freund
Hi, On 2025-02-16 10:47:40 -0800, Noah Misch wrote: > On Sun, Feb 16, 2025 at 01:02:01PM -0500, Andres Freund wrote: > > On 2025-02-16 09:39:43 -0800, Noah Misch wrote: > > > On Thu, Feb 13, 2025 at 12:39:04PM -0500, Andres Freund wrote: > > > > I suspect what's happening is that the communication

Re: BackgroundPsql swallowing errors on windows

2025-02-16 Thread Noah Misch
On Sun, Feb 16, 2025 at 01:02:01PM -0500, Andres Freund wrote: > On 2025-02-16 09:39:43 -0800, Noah Misch wrote: > > On Thu, Feb 13, 2025 at 12:39:04PM -0500, Andres Freund wrote: > > > I suspect what's happening is that the communication with the > > > external process allows for reordering betwee

Re: BackgroundPsql swallowing errors on windows

2025-02-16 Thread Andres Freund
Hi, On 2025-02-16 09:39:43 -0800, Noah Misch wrote: > On Thu, Feb 13, 2025 at 12:39:04PM -0500, Andres Freund wrote: > > I suspect what's happening is that the communication with the > > external process allows for reordering between stdout/stderr. > > > > And indeed, changing BackgroundPsql::que

Re: BackgroundPsql swallowing errors on windows

2025-02-16 Thread Noah Misch
On Thu, Feb 13, 2025 at 12:39:04PM -0500, Andres Freund wrote: > My understanding is that IPC::Run uses a proxy process on windows to execute > subprocesses and then communicates with that over TCP (or something along > those lines). Right. > I suspect what's happening is that the communication w

Re: BackgroundPsql swallowing errors on windows

2025-02-15 Thread Andres Freund
Hi, On 2025-02-14 09:52:24 -0800, Jacob Champion wrote: > On Fri, Feb 14, 2025 at 8:53 AM Andres Freund wrote: > > commit 70291a3c66e > > Author: Michael Paquier > > Date: 2024-11-07 12:11:27 +0900 > > > > Improve handling of empty query results in BackgroundPsql::query() > > > > commit ba

Re: BackgroundPsql swallowing errors on windows

2025-02-14 Thread Andres Freund
Hi, On 2025-02-14 12:14:55 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2025-02-14 08:14:45 -0500, Andrew Dunstan wrote: > >> ... If there is interest I will > >> bring the work up to date, and maybe we can introduce this early in the v19 > >> cycle. It would significantly reduce our dep

Re: BackgroundPsql swallowing errors on windows

2025-02-14 Thread Jacob Champion
On Fri, Feb 14, 2025 at 8:53 AM Andres Freund wrote: > commit 70291a3c66e > Author: Michael Paquier > Date: 2024-11-07 12:11:27 +0900 > > Improve handling of empty query results in BackgroundPsql::query() > > commit ba08edb0654 > Author: Michael Paquier > Date: 2024-11-06 15:31:14 +0900

Re: BackgroundPsql swallowing errors on windows

2025-02-14 Thread Tom Lane
Andres Freund writes: > On 2025-02-14 08:14:45 -0500, Andrew Dunstan wrote: >> ... If there is interest I will >> bring the work up to date, and maybe we can introduce this early in the v19 >> cycle. It would significantly reduce our dependency on IPC::Run, especially >> the pump() stuff. > I def

Re: BackgroundPsql swallowing errors on windows

2025-02-14 Thread Andres Freund
Hi, On 2025-02-14 13:35:40 +0100, Daniel Gustafsson wrote: > > On 13 Feb 2025, at 18:39, Andres Freund wrote: > > > The banner being the same between queries made it hard to understand if a > > banner that appeared in the output was from the current query or a past > > query. Therefore I added a

Re: BackgroundPsql swallowing errors on windows

2025-02-14 Thread Andres Freund
Hi, On 2025-02-14 08:14:45 -0500, Andrew Dunstan wrote: > It's been discussed before, but I'd really really like to get rid of > BackgroundPsql. It's ugly, non-intuitive and fragile. I agree, unfortunately we're stuck with this until we have a better alternative in tree :( > Last year I did som

Re: BackgroundPsql swallowing errors on windows

2025-02-14 Thread Andrew Dunstan
On 2025-02-13 Th 12:39 PM, Andres Freund wrote: Hi, One of the remaining tasks for AIO was to convert the tests to be proper tap tests. I did that and was thanked by the tests fairly randomly failing on windows. Which test fails changes from run to run. The symptom is that BackgroundPsql->qu

Re: BackgroundPsql swallowing errors on windows

2025-02-14 Thread Daniel Gustafsson
> On 13 Feb 2025, at 18:39, Andres Freund wrote: > The banner being the same between queries made it hard to understand if a > banner that appeared in the output was from the current query or a past > query. Therefore I added a counter to it. + my $banner = "background_psql: QUERY_SEPARATO

BackgroundPsql swallowing errors on windows

2025-02-13 Thread Andres Freund
Hi, One of the remaining tasks for AIO was to convert the tests to be proper tap tests. I did that and was thanked by the tests fairly randomly failing on windows. Which test fails changes from run to run. The symptom is that BackgroundPsql->query() sometimes would simply swallow errors that wer