Re: bad wal on replica / incorrect resource manager data checksum in record / zfs

2020-04-06 Thread Alex Malek
On Thu, Apr 2, 2020 at 2:10 PM Andres Freund wrote: > Hi, > > On 2020-02-19 16:35:53 -0500, Alex Malek wrote: > > We are having a reoccurring issue on 2 of our replicas where replication > > stops due to this message: > > "incorrect resource manager data checksum in record at ..." > > Could you s

Re: bad wal on replica / incorrect resource manager data checksum in record / zfs

2020-04-02 Thread Andres Freund
Hi, On 2020-02-19 16:35:53 -0500, Alex Malek wrote: > We are having a reoccurring issue on 2 of our replicas where replication > stops due to this message: > "incorrect resource manager data checksum in record at ..." Could you show the *exact* log output please? Because this could temporarily oc

Re: bad wal on replica / incorrect resource manager data checksum in record / zfs

2020-04-02 Thread Alex Malek
On Wed, Feb 19, 2020 at 4:35 PM Alex Malek wrote: > > Hello Postgres Hackers - > > We are having a reoccurring issue on 2 of our replicas where replication > stops due to this message: > "incorrect resource manager data checksum in record at ..." > This has been occurring on average once every 1

Re: bad wal on replica / incorrect resource manager data checksum in record / zfs

2020-02-26 Thread Alex Malek
On Thu, Feb 20, 2020 at 12:01 PM Alex Malek wrote: > On Thu, Feb 20, 2020, 6:16 AM Amit Kapila wrote: > >> On Thu, Feb 20, 2020 at 3:06 AM Alex Malek wrote: >> > >> > >> > Hello Postgres Hackers - >> > >> > We are having a reoccurring issue on 2 of our replicas where >> replication stops due to

Re: bad wal on replica / incorrect resource manager data checksum in record / zfs

2020-02-20 Thread Amit Kapila
On Thu, Feb 20, 2020 at 7:40 PM Alex Malek wrote: > > On Thu, Feb 20, 2020, 6:16 AM Amit Kapila wrote: >> >> On Thu, Feb 20, 2020 at 3:06 AM Alex Malek wrote: >> > >> > Some interesting data points while debugging: >> > We had lowered the ZFS recordsize from 128K to 32K and for that week the >>

Fwd: bad wal on replica / incorrect resource manager data checksum in record / zfs

2020-02-20 Thread Alex Malek
On Thu, Feb 20, 2020, 6:16 AM Amit Kapila wrote: > On Thu, Feb 20, 2020 at 3:06 AM Alex Malek wrote: > > > > > > Hello Postgres Hackers - > > > > We are having a reoccurring issue on 2 of our replicas where replication > stops due to this message: > > "incorrect resource manager data checksum in

Re: bad wal on replica / incorrect resource manager data checksum in record / zfs

2020-02-20 Thread Amit Kapila
On Thu, Feb 20, 2020 at 3:06 AM Alex Malek wrote: > > > Hello Postgres Hackers - > > We are having a reoccurring issue on 2 of our replicas where replication > stops due to this message: > "incorrect resource manager data checksum in record at ..." > This has been occurring on average once every

bad wal on replica / incorrect resource manager data checksum in record / zfs

2020-02-19 Thread Alex Malek
Hello Postgres Hackers - We are having a reoccurring issue on 2 of our replicas where replication stops due to this message: "incorrect resource manager data checksum in record at ..." This has been occurring on average once every 1 to 2 weeks during large data imports (100s of GBs being written)