Hello Adrian, thanks for the response. master data also located on SAN
Yes, each replica is it own VM with its own virtual disk/volume as served up from the same SAN Raw disk mappings are a way for ESX to present a SAN volume directly to a VM instead of creating a virtual disk. no unexpected messages detected. On Sun, Apr 3, 2016 at 11:23 PM, Adrian Klaver <adrian.kla...@aklaver.com> wrote: > On 04/02/2016 08:38 PM, Soni M wrote: > >> Hello Everyone, >> >> We face TOAST table corruption. >> >> One master and two streaming replicas. The corruption happen only on >> both streaming replicas. >> >> We did found the corrupted rows. Selecting on this row, return (on both >> replica) : unexpected chunk number 0 (expected 1) for toast value >> 1100613112 in pg_toast_112517 >> selecting this row on master does not return corruption error, but >> return correct result instead. >> >> Previously, dump on a replica return : unexpected chunk number 0 >> (expected 1) for toast value 3234098599 in pg_toast_112517 (please note >> the toast value is different) >> >> This table size is 343 GB, contain around 206,179,697 live tuples. We >> found that the corruption happen on the biggest column (this column and >> its pkey sized around 299 GB total). >> >> > on both replica : >> fsync NEVER turned off. >> none unexpected power loss nor OS crash. >> >> How can the corruption occurs ? and how can I resolve them ? >> > > Meant to add to previous post. > > Do you see anything in the replica Postgres logs that indicate a problem > with the replication process? > > Or any other unexpected messages prior to the point you did the select on > the replica(s)? > > > >> Thank so much for the help. >> >> Cheers \o/ >> >> -- >> Regards, >> >> Soni Maula Harriz >> > > > -- > Adrian Klaver > adrian.kla...@aklaver.com > -- Regards, Soni Maula Harriz
