Re: deferred writing of two-phase state files adds fragility

2024-12-05 Thread Andres Freund
Hi, On 2024-12-05 11:21:12 -0500, Robert Haas wrote: > On Wed, Dec 4, 2024 at 6:36 PM Andres Freund wrote: > > How did this corruption actually come about? Did it actually really just > > affect that single WAL segment? Somehow that doesn't seem too likely. > > I don't know and might not be abl

Re: deferred writing of two-phase state files adds fragility

2024-12-05 Thread Robert Haas
On Wed, Dec 4, 2024 at 6:36 PM Andres Freund wrote: > Is 2PC really that special in that regard? If the WAL that contains the > checkpoint record itself gets corrupted, you're also in a world of hurt, once > you shut down? Or, to a slightly lower degree, if there's any corrupted > record between

Re: deferred writing of two-phase state files adds fragility

2024-12-04 Thread Andres Freund
Hi, On 2024-12-04 12:04:47 -0500, Robert Haas wrote: > Let's suppose that you execute PREPARE TRANSACTION and, before the > next CHECKPOINT, the WAL record for the PREPARE TRANSACTION gets > corrupted on disk. This might seem like an unlikely scenario, and it > is, but we saw a case at EDB not too

Re: deferred writing of two-phase state files adds fragility

2024-12-04 Thread Tom Lane
Robert Haas writes: > On Wed, Dec 4, 2024 at 12:19 PM Tom Lane wrote: >> Could we fix it so ROLLBACK TRANSACTION removes the GID from the >> list of prepared xacts that need to be written out? Then we'd >> no longer have a pending requirement to read the broken WAL record. > That would be nice,

Re: deferred writing of two-phase state files adds fragility

2024-12-04 Thread Robert Haas
On Wed, Dec 4, 2024 at 12:58 PM Tom Lane wrote: > Robert Haas writes: > > On Wed, Dec 4, 2024 at 12:19 PM Tom Lane wrote: > >> Could we fix it so ROLLBACK TRANSACTION removes the GID from the > >> list of prepared xacts that need to be written out? Then we'd > >> no longer have a pending requir

Re: deferred writing of two-phase state files adds fragility

2024-12-04 Thread Robert Haas
On Wed, Dec 4, 2024 at 12:19 PM Tom Lane wrote: > > You can't execute COMMIT > > TRANSACTION or ROLLBACK TRANSACTION, so there's now way to resolve the > > prepared transaction. > > Could we fix it so ROLLBACK TRANSACTION removes the GID from the > list of prepared xacts that need to be written ou

Re: deferred writing of two-phase state files adds fragility

2024-12-04 Thread Tom Lane
Robert Haas writes: > Let's suppose that you execute PREPARE TRANSACTION and, before the > next CHECKPOINT, the WAL record for the PREPARE TRANSACTION gets > corrupted on disk. This might seem like an unlikely scenario, and it > is, but we saw a case at EDB not too long ago. > To a first approxim

deferred writing of two-phase state files adds fragility

2024-12-04 Thread Robert Haas
Let's suppose that you execute PREPARE TRANSACTION and, before the next CHECKPOINT, the WAL record for the PREPARE TRANSACTION gets corrupted on disk. This might seem like an unlikely scenario, and it is, but we saw a case at EDB not too long ago. To a first approximation, the world ends. You can'