Re: Enable data checksums by default

2025-06-06 Thread Bruce Momjian
On Fri, Jun 6, 2025 at 01:40:40PM +0200, Peter Eisentraut wrote: > On 05.06.25 12:37, Christoph Berg wrote: > > Maybe it would be enough if the initdb options used to create a > > cluster would be stored in some file in the data dir. > > That probably wouldn't help if the default behavior changed

Re: Enable data checksums by default

2025-06-06 Thread Peter Eisentraut
On 05.06.25 12:37, Christoph Berg wrote: Maybe it would be enough if the initdb options used to create a cluster would be stored in some file in the data dir. That probably wouldn't help if the default behavior changed, as in the current case.

Re: Enable data checksums by default

2025-06-05 Thread Christoph Berg
Re: Heikki Linnakangas > 1. Have pg_upgrade run initdb for you. It's always felt silly that you need > to run initdb with the new version yourself, when there's really only one > correct way to do it. Fwiw, Debian's pg_upgradecluster script would be happy if that problem would be solved. Currently

Re: Enable data checksums by default

2025-06-04 Thread Peter Eisentraut
On 23.05.25 11:43, Tomas Vondra wrote: We don't currently have anything in the release notes that calls this out as a potential upgrading issue, so I propose the attached patch. Seems reasonable, although maybe it should say ... so if the old cluster does not have checksums enabled ... ins

Re: Enable data checksums by default

2025-05-23 Thread Bruce Momjian
On Fri, May 23, 2025 at 11:10:47AM +0300, Heikki Linnakangas wrote: > On 24/04/2025 14:26, Peter Eisentraut wrote: > > action for beta1 is (a).  And then for (c) perhaps monitor the feedback > > between beta1 and beta2. > > > > Ping: It's time to do something about this open item. (Or decide to d

Re: Enable data checksums by default

2025-05-23 Thread Daniel Gustafsson
> On 23 May 2025, at 10:10, Heikki Linnakangas wrote: > Aside from just documenting it, I see two things we could do: > > 1. Have pg_upgrade run initdb for you. It's always felt silly that you need > to run initdb with the new version yourself, when there's really only one > correct way to do

Re: Enable data checksums by default

2025-05-23 Thread Daniel Gustafsson
> On 23 May 2025, at 11:55, Tomas Vondra wrote: > > On 5/23/25 11:25, Daniel Gustafsson wrote: >>> On 23 May 2025, at 10:10, Heikki Linnakangas wrote: >> >>> Aside from just documenting it, I see two things we could do: >>> >>> 1. Have pg_upgrade run initdb for you. It's always felt silly that

Re: Enable data checksums by default

2025-05-23 Thread Peter Eisentraut
On 23.05.25 10:10, Heikki Linnakangas wrote: The point of the open item was (a) to make sure this is adequately documented, for instance in the release notes, (b) to think about technological solutions to simplify this, such as [0], and (c) to just check the general feedback. Nothing from [0]

Re: Enable data checksums by default

2025-05-23 Thread Tomas Vondra
On 5/23/25 11:25, Daniel Gustafsson wrote: >> On 23 May 2025, at 10:10, Heikki Linnakangas wrote: > >> Aside from just documenting it, I see two things we could do: >> >> 1. Have pg_upgrade run initdb for you. It's always felt silly that you need >> to run initdb with the new version yourself, w

Re: Enable data checksums by default

2025-05-23 Thread Tomas Vondra
On 5/23/25 11:22, Peter Eisentraut wrote: > On 23.05.25 10:10, Heikki Linnakangas wrote: >>> The point of the open item was (a) to make sure this is adequately >>> documented, for instance in the release notes, (b) to think about >>> technological solutions to simplify this, such as [0], and (c)

Re: Enable data checksums by default

2025-05-23 Thread Heikki Linnakangas
On 24/04/2025 14:26, Peter Eisentraut wrote: On 23.04.25 00:24, Tomas Vondra wrote: The patch that flips the default has been committed. I also started a PG18 open items page and made a note that we follow up on the upgrade experience, as was discussed in this thread. https://wiki.postgresql.o

Re: Enable data checksums by default

2025-04-24 Thread Peter Eisentraut
On 23.04.25 00:24, Tomas Vondra wrote: The patch that flips the default has been committed. I also started a PG18 open items page and made a note that we follow up on the upgrade experience, as was discussed in this thread. https://wiki.postgresql.org/wiki/PostgreSQL_18_Open_Items Regarding t

Re: Enable data checksums by default

2025-04-23 Thread Christoph Berg
Re: Tomas Vondra > We went through the open items on the RMT team meeting today, and my > impression was the questions are mostly about performance of having > checksums by default, but now I realize the thread talks about "upgrade > experience" which seems fairly wide. Fwiw, Debian's pg_upgradecl

Re: Enable data checksums by default

2025-04-22 Thread Tomas Vondra
On 10/16/24 08:54, Peter Eisentraut wrote: > On 14.10.24 11:28, Peter Eisentraut wrote: >> On 03.10.24 23:13, Nathan Bossart wrote: >>> On Tue, Oct 01, 2024 at 11:15:02AM -0400, Peter Eisentraut wrote: I have committed 0001 (the new option) and 0004 (the docs tweak).  I think there i

Re: Enable data checksums by default

2024-11-13 Thread Alvaro Herrera
On 2024-Nov-13, Peter Eisentraut wrote: > On 07.11.24 19:38, Alvaro Herrera wrote: > > I have just noticed that since this patch was committed as 04bec894a04c, > > pg_upgrade's "make check" action is unusable when given the > > "olddump/oldinstall" options. We now need to inject '-k' to the initd

Re: Enable data checksums by default

2024-11-13 Thread Peter Eisentraut
On 07.11.24 19:38, Alvaro Herrera wrote: I have just noticed that since this patch was committed as 04bec894a04c, pg_upgrade's "make check" action is unusable when given the "olddump/oldinstall" options. We now need to inject '-k' to the initdb line for old servers, and we don't, so all upgrade

Re: Enable data checksums by default

2024-11-07 Thread Alvaro Herrera
I have just noticed that since this patch was committed as 04bec894a04c, pg_upgrade's "make check" action is unusable when given the "olddump/oldinstall" options. We now need to inject '-k' to the initdb line for old servers, and we don't, so all upgrade tests fail. I think this patch should be e

Re: Enable data checksums by default

2024-10-16 Thread Daniel Gustafsson
> On 16 Oct 2024, at 09:49, Peter Eisentraut wrote: > Ah yes, and the upgrade tests on the buildfarm don't like this. What shall > we do about this? Maybe adjust the buildfarm scripts to use > --no-data-checksums? I can't see many other options, and it represents a reasonable use-case, so +1

Re: Enable data checksums by default

2024-10-16 Thread Peter Eisentraut
On 16.10.24 08:54, Peter Eisentraut wrote: On 14.10.24 11:28, Peter Eisentraut wrote: On 03.10.24 23:13, Nathan Bossart wrote: On Tue, Oct 01, 2024 at 11:15:02AM -0400, Peter Eisentraut wrote: I have committed 0001 (the new option) and 0004 (the docs tweak).  I think there is consensus for the

Re: Enable data checksums by default

2024-10-15 Thread Peter Eisentraut
On 14.10.24 11:28, Peter Eisentraut wrote: On 03.10.24 23:13, Nathan Bossart wrote: On Tue, Oct 01, 2024 at 11:15:02AM -0400, Peter Eisentraut wrote: I have committed 0001 (the new option) and 0004 (the docs tweak).  I think there is consensus for the rest, too, but I'll leave it for a few mor

Re: Enable data checksums by default

2024-10-14 Thread Peter Eisentraut
On 03.10.24 23:13, Nathan Bossart wrote: On Tue, Oct 01, 2024 at 11:15:02AM -0400, Peter Eisentraut wrote: I have committed 0001 (the new option) and 0004 (the docs tweak). I think there is consensus for the rest, too, but I'll leave it for a few more days to think about. I guess the test fail

Re: Enable data checksums by default

2024-10-07 Thread Daniel Gustafsson
> On 3 Oct 2024, at 23:13, Nathan Bossart wrote: > > On Tue, Oct 01, 2024 at 11:15:02AM -0400, Peter Eisentraut wrote: >> I have committed 0001 (the new option) and 0004 (the docs tweak). I think >> there is consensus for the rest, too, but I'll leave it for a few more days >> to think about. I

Re: Enable data checksums by default

2024-10-03 Thread Nathan Bossart
On Tue, Oct 01, 2024 at 11:15:02AM -0400, Peter Eisentraut wrote: > I have committed 0001 (the new option) and 0004 (the docs tweak). I think > there is consensus for the rest, too, but I'll leave it for a few more days > to think about. I guess the test failure has to be addressed. Here is a re

Re: Enable data checksums by default

2024-10-01 Thread Peter Eisentraut
On 27.08.24 17:26, Nathan Bossart wrote: On Tue, Aug 27, 2024 at 05:16:51PM +0200, Peter Eisentraut wrote: On 27.08.24 15:44, Greg Sabino Mullane wrote: On Mon, Aug 26, 2024 at 3:46 PM Nathan Bossart mailto:nathandboss...@gmail.com>> wrote: Should we error if both --data-checksum and --no

Re: Enable data checksums by default

2024-08-27 Thread Nathan Bossart
On Tue, Aug 27, 2024 at 05:16:51PM +0200, Peter Eisentraut wrote: > On 27.08.24 15:44, Greg Sabino Mullane wrote: >> On Mon, Aug 26, 2024 at 3:46 PM Nathan Bossart > > wrote: >> >> Should we error if both --data-checksum and --no-data-checksums are >> speci

Re: Enable data checksums by default

2024-08-27 Thread Peter Eisentraut
On 27.08.24 15:44, Greg Sabino Mullane wrote: On Mon, Aug 26, 2024 at 3:46 PM Nathan Bossart > wrote: Should we error if both --data-checksum and --no-data-checksums are specified?  IIUC with 0001, we'll use whichever is specified last. Hmmm, that is a

Re: Enable data checksums by default

2024-08-27 Thread Greg Sabino Mullane
On Mon, Aug 26, 2024 at 3:46 PM Nathan Bossart wrote: > Should we error if both --data-checksum and --no-data-checksums are > specified? IIUC with 0001, we'll use whichever is specified last. > Hmmm, that is a good question. We have never (to my recollection) flipped a default quite like this b

Re: Enable data checksums by default

2024-08-26 Thread Nathan Bossart
In general, +1 for $SUBJECT. printf(_(" -k, --data-checksums use data page checksums\n")); +printf(_(" --no-data-checksums do not use data page checksums\n")); Should we error if both --data-checksum and --no-data-checksums are specified? IIUC with 0001, we'll use whichever

Re: Enable data checksums by default

2024-08-23 Thread Bruce Momjian
On Fri, Aug 23, 2024 at 03:17:17PM +0200, Peter Eisentraut wrote: > On 08.08.24 19:19, Greg Sabino Mullane wrote: > > Thank you for the feedback. Please find attached three separate patches. > > One to add a new flag to initdb (--no-data-checksums), one to adjust the > > tests to use this flag as n

Re: Enable data checksums by default

2024-08-23 Thread Greg Sabino Mullane
Rebased and reworked patches attached: v2-0001-Add-new-initdb-argument-no-data-checksums-to-force-checksums-off.patch - Adds --no-data-checksums to initdb.c, adjusts --help and sgml docs, adds simple tests v2-0002-Allow-tests-to-force-checksums-off-when-calling-init.patch - Adjusts the Perl tests

Re: Enable data checksums by default

2024-08-23 Thread Peter Eisentraut
On 08.08.24 19:19, Greg Sabino Mullane wrote: Thank you for the feedback. Please find attached three separate patches. One to add a new flag to initdb (--no-data-checksums), one to adjust the tests to use this flag as needed, and the final to make the actual switch of the default value (along w

Re: Enable data checksums by default

2024-08-22 Thread Jakub Wartak
On Thu, Aug 22, 2024 at 8:11 AM Peter Eisentraut wrote: > > On 15.08.24 08:38, Peter Eisentraut wrote: > > On 08.08.24 19:42, Robert Haas wrote: > >>> I'm thinking pg_upgrade could have a mode where it adds the > >>> checksum during the upgrade as it copies the files (essentially a subset > >>> of

Re: Enable data checksums by default

2024-08-21 Thread Peter Eisentraut
On 15.08.24 08:38, Peter Eisentraut wrote: On 08.08.24 19:42, Robert Haas wrote: I'm thinking pg_upgrade could have a mode where it adds the checksum during the upgrade as it copies the files (essentially a subset of pg_checksums).  I think that would be useful for that middle tier of users who

Re: Enable data checksums by default

2024-08-15 Thread Jakub Wartak
Hi all, On Tue, Aug 13, 2024 at 10:08 PM Robert Haas wrote: > And it's not like we have statistics anywhere that you can look at to > see how much CPU time you spent computing checksums, so if a user DOES > have a performance problem that would not have occurred if checksums > had been disabled,

Re: Enable data checksums by default

2024-08-15 Thread Michael Banck
On Thu, Aug 15, 2024 at 09:49:04AM +0200, Jakub Wartak wrote: > On Wed, Aug 7, 2024 at 4:18 PM Greg Sabino Mullane wrote: > > > > On Wed, Aug 7, 2024 at 4:43 AM Michael Banck wrote: > >> > >> I think the last time we dicussed this the consensus was that > >> computational overhead of computing th

Re: Enable data checksums by default

2024-08-15 Thread Jakub Wartak
Hi Greg and others On Tue, Aug 13, 2024 at 4:42 PM Greg Sabino Mullane wrote: > > On Thu, Aug 8, 2024 at 6:11 AM Peter Eisentraut wrote: > >> >> My understanding was that the reason for some hesitation about adopting data >> checksums was the performance impact. Not the checksumming itself, bu

Re: Enable data checksums by default

2024-08-15 Thread Jakub Wartak
On Wed, Aug 7, 2024 at 4:18 PM Greg Sabino Mullane wrote: > > On Wed, Aug 7, 2024 at 4:43 AM Michael Banck wrote: >> >> I think the last time we dicussed this the consensus was that >> computational overhead of computing the checksums is pretty small for >> most systems (so the above change seems

Re: Enable data checksums by default

2024-08-14 Thread Peter Eisentraut
On 08.08.24 19:42, Robert Haas wrote: I'm thinking pg_upgrade could have a mode where it adds the checksum during the upgrade as it copies the files (essentially a subset of pg_checksums). I think that would be useful for that middle tier of users who just want a good default experience. That w

Re: Enable data checksums by default

2024-08-13 Thread Robert Haas
On Tue, Aug 13, 2024 at 10:42 AM Greg Sabino Mullane wrote: > Fair enough. I think the performance impact is acceptable, as evidenced by > the large number of people that turn it on. And it is easy enough to turn it > off again, either via --no-data-checksums or pg_checksums --disable. > When I

Re: Enable data checksums by default

2024-08-13 Thread Greg Sabino Mullane
On Thu, Aug 8, 2024 at 6:11 AM Peter Eisentraut wrote: > My understanding was that the reason for some hesitation about adopting > data checksums was the performance impact. Not the checksumming itself, > but the overhead from hint bit logging. The last time I looked into that, > you could get

Re: Enable data checksums by default

2024-08-08 Thread Tomas Vondra
On 8/8/24 19:42, Robert Haas wrote: > On Thu, Aug 8, 2024 at 6:11 AM Peter Eisentraut wrote: >> About the claim that it's already the de-facto standard. Maybe that is >> approximately true for "serious" installations. But AFAICT, the popular >> packagings don't enable checksums by default, so th

Re: Enable data checksums by default

2024-08-08 Thread Robert Haas
On Thu, Aug 8, 2024 at 6:11 AM Peter Eisentraut wrote: > About the claim that it's already the de-facto standard. Maybe that is > approximately true for "serious" installations. But AFAICT, the popular > packagings don't enable checksums by default, so there is likely a > significant middle tier

Re: Enable data checksums by default

2024-08-08 Thread Greg Sabino Mullane
Thank you for the feedback. Please find attached three separate patches. One to add a new flag to initdb (--no-data-checksums), one to adjust the tests to use this flag as needed, and the final to make the actual switch of the default value (along with tests and docs). Cheers, Greg 0001-Add-new-

Re: Enable data checksums by default

2024-08-08 Thread Daniel Gustafsson
> On 8 Aug 2024, at 12:11, Peter Eisentraut wrote: > My understanding was that the reason for some hesitation about adopting data > checksums was the performance impact. Not the checksumming itself, but the > overhead from hint bit logging. The last time I looked into that, you could > get p

Re: Enable data checksums by default

2024-08-08 Thread Michael Banck
On Thu, Aug 08, 2024 at 12:11:38PM +0200, Peter Eisentraut wrote: > So I think we need to think through the upgrade experience a bit more. > Unfortunately, pg_checksums hasn't gotten to the point that we were perhaps > once hoping for that you could enable checksums on a live system. I'm > thinkin

Re: Enable data checksums by default

2024-08-08 Thread Peter Eisentraut
On 07.08.24 00:46, Greg Sabino Mullane wrote: Currently, initdb only enables data checksums if passed the --data-checksums or -k argument. There was some hesitation years ago when this feature was first added, leading to the current situation where the default is off. However, many years later,

Re: Enable data checksums by default

2024-08-07 Thread Michael Paquier
On Aug 7, 2024, at 23:18, Greg Sabino Mullane wrote:On Wed, Aug 7, 2024 at 4:43 AM Michael Banck wrote:. Does it make sense to add -K (capital k) as a short-cut for this? I think this is how we distinguish on/off for pg_dump (-t/-T etc.) but maybe that is not wider project policy.

Re: Enable data checksums by default

2024-08-07 Thread Greg Sabino Mullane
On Wed, Aug 7, 2024 at 4:43 AM Michael Banck wrote: > I think the last time we dicussed this the consensus was that > computational overhead of computing the checksums is pretty small for > most systems (so the above change seems warranted regardless of whether > we switch the default), but turni

Re: Enable data checksums by default

2024-08-07 Thread Michael Banck
Hi, On Tue, Aug 06, 2024 at 06:46:52PM -0400, Greg Sabino Mullane wrote: > Please find attached a patch to enable data checksums by default. > > Currently, initdb only enables data checksums if passed the > --data-checksums or -k argument. There was some hesitation years ago when &g

Enable data checksums by default

2024-08-06 Thread Greg Sabino Mullane
Please find attached a patch to enable data checksums by default. Currently, initdb only enables data checksums if passed the --data-checksums or -k argument. There was some hesitation years ago when this feature was first added, leading to the current situation where the default is off. However

Re: Enable data checksums by default

2019-04-11 Thread Daniel Gustafsson
On Thursday, April 11, 2019 8:56 PM, Andres Freund wrote: > On 2019-04-11 18:15:41 +, Daniel Gustafsson wrote: > > > On Thursday, April 11, 2019 6:58 PM, Andres Freund and...@anarazel.de wrote: > > > > > On 2019-04-09 23:11:03 -0400, Bruce Momjian wrote: > > > > > > > Enabling checksums by de

Re: Enable data checksums by default

2019-04-11 Thread Andres Freund
Hi, On 2019-04-11 18:15:41 +, Daniel Gustafsson wrote: > On Thursday, April 11, 2019 6:58 PM, Andres Freund wrote: > > > On 2019-04-09 23:11:03 -0400, Bruce Momjian wrote: > > > > > Enabling checksums by default will require anyone using pg_upgrade to > > > run initdb to disable checksums be

Re: Enable data checksums by default

2019-04-11 Thread Daniel Gustafsson
On Thursday, April 11, 2019 6:58 PM, Andres Freund wrote: > On 2019-04-09 23:11:03 -0400, Bruce Momjian wrote: > > > Enabling checksums by default will require anyone using pg_upgrade to > > run initdb to disable checksums before running pg_upgrade, for one > > release. We could add checksums for

Re: Enable data checksums by default

2019-04-11 Thread Andres Freund
Hi, On 2019-04-09 23:11:03 -0400, Bruce Momjian wrote: > Enabling checksums by default will require anyone using pg_upgrade to > run initdb to disable checksums before running pg_upgrade, for one > release. We could add checksums for non-link pg_upgrade runs, but we > don't have code to do that y

Re: Enable data checksums by default

2019-04-09 Thread Bruce Momjian
On Fri, Mar 22, 2019 at 12:07:22PM -0400, Tom Lane wrote: > Christoph Berg writes: > > I think, the next step in that direction would be to enable data > > checksums by default. They make sense in most setups, > > Well, that is exactly the point that needs some proof, no

Re: Enable data checksums by default

2019-04-09 Thread Bruce Momjian
AL all the time, and there's not even an option to disable it, > even if that might make things faster. Why don't we enable data > checksums by default as well? We checksum wal because we know partial WAL writes are likely to happen during power failure during a write. Data pages h

Re: Enable data checksums by default

2019-04-01 Thread Magnus Hagander
the time, and there's not even an option to disable it, > even if that might make things faster. Why don't we enable data > checksums by default as well? > I think one of the often overlooked original reasons was that we need to log hint bits, same as when wal_log_hints is set. Of cou

Re: Enable data checksums by default

2019-04-01 Thread Christoph Berg
sage. Thanks for doing these tests. The point I'm trying to make is, why do we run without data checksums by default? For example, we do checksum the WAL all the time, and there's not even an option to disable it, even if that might make things faster. Why don't we enable data checksums by default as well? Christoph

Re: Enable data checksums by default

2019-03-30 Thread Andres Freund
On March 30, 2019 3:25:43 PM EDT, Tomas Vondra wrote: >On Fri, Mar 29, 2019 at 08:35:26PM +0100, Christoph Berg wrote: >>Re: Bernd Helmle 2019-03-29 ><3586bb9345a59bfc8d13a50a7c729be1ee6759fd.ca...@oopsware.de> >>> Am Freitag, den 29.03.2019, 23:10 +0900 schrieb Michael Paquier: >>> > >>> > I

Re: Enable data checksums by default

2019-03-30 Thread Tomas Vondra
On Fri, Mar 29, 2019 at 08:35:26PM +0100, Christoph Berg wrote: Re: Bernd Helmle 2019-03-29 <3586bb9345a59bfc8d13a50a7c729be1ee6759fd.ca...@oopsware.de> Am Freitag, den 29.03.2019, 23:10 +0900 schrieb Michael Paquier: > > I can't really believe that many people set up shared_buffers at > 128kB

Re: Enable data checksums by default

2019-03-29 Thread Christoph Berg
Re: Bernd Helmle 2019-03-29 <3586bb9345a59bfc8d13a50a7c729be1ee6759fd.ca...@oopsware.de> > Am Freitag, den 29.03.2019, 23:10 +0900 schrieb Michael Paquier: > > > > I can't really believe that many people set up shared_buffers at > > 128kB > > which would cause such a large number of page eviction

Re: Enable data checksums by default

2019-03-29 Thread Bernd Helmle
Am Freitag, den 29.03.2019, 23:10 +0900 schrieb Michael Paquier: > > I can't really believe that many people set up shared_buffers at > 128kB > which would cause such a large number of page evictions, but I can > believe that many users have shared_buffers set to its default value > and that we ar

Re: Enable data checksums by default

2019-03-29 Thread Michael Paquier
On Fri, Mar 29, 2019 at 11:16:11AM +0100, Bernd Helmle wrote: > So between ~7% to 18% impact with checksums in this specific case here. I can't really believe that many people set up shared_buffers at 128kB which would cause such a large number of page evictions, but I can believe that many users

Re: Enable data checksums by default

2019-03-29 Thread Ants Aasma
On Thu, Mar 28, 2019 at 10:38 AM Christoph Berg wrote: > Re: Ants Aasma 2019-03-27 < > ca+csw_twxdrzdn2xsszbxej63dez+f6_hs3qf7hmxfenxsq...@mail.gmail.com> > > Can you try with postgres compiled with CFLAGS="-O2 -march=native"? > There's > > a bit of low hanging fruit there to use a runtime CPU ch

Re: Enable data checksums by default

2019-03-29 Thread Bernd Helmle
Am Dienstag, den 26.03.2019, 16:14 +0100 schrieb Christoph Berg: > select 92551.0/97363; > 0.9506 > > So the cost is 5% in this very contrived case. In almost any other > setting, the cost would be lower, I'd think. Well, my machine (Intel(R) Core(TM) i7-6770HQ CPU @ 2.60GHz, 32 GByte RAM) tells

Re: Enable data checksums by default

2019-03-28 Thread Peter Eisentraut
On 2019-03-22 16:16, Christoph Berg wrote: > I think, the next step in that direction would be to enable data > checksums by default. They make sense in most setups, and people who > plan to run very performance-critical systems where checksums might be > too much need to tune many

Re: Enable data checksums by default

2019-03-28 Thread Christoph Berg
Re: Ants Aasma 2019-03-27 > Can you try with postgres compiled with CFLAGS="-O2 -march=native"? There's > a bit of low hanging fruit there to use a runtime CPU check to pick a > better optimized checksum function. Frankly, no. This is with the apt.pg.o packages which are supposed to be usable by

Re: Enable data checksums by default

2019-03-27 Thread Ants Aasma
On Wed, Mar 27, 2019, 15:57 Christoph Berg wrote: > Re: To Tom Lane 2019-03-26 <20190326151446.gg3...@msg.df7cb.de> > > I run a benchmark with checksums disabled/enabled. shared_buffers is > > 512kB to make sure almost any read will fetch the page from the OS > > cache; scale factor is 50 (~750MB

Re: Enable data checksums by default

2019-03-27 Thread Christoph Berg
Re: To Tom Lane 2019-03-26 <20190326151446.gg3...@msg.df7cb.de> > I run a benchmark with checksums disabled/enabled. shared_buffers is > 512kB to make sure almost any read will fetch the page from the OS > cache; scale factor is 50 (~750MB) to make sure the whole cluster fits > into RAM. [...] > So

Re: Enable data checksums by default

2019-03-26 Thread Peter Geoghegan
On Fri, Mar 22, 2019 at 9:07 AM Tom Lane wrote: > IMO, the main value of checksums is that they allow the Postgres > project to deflect blame. That's nice for us but I'm not sure > that it's a benefit for users. I've seen little if any data to > suggest that checksums actually catch enough probl

Re: Enable data checksums by default

2019-03-26 Thread Christoph Berg
Re: Tom Lane 2019-03-22 <4368.1553270...@sss.pgh.pa.us> > Christoph Berg writes: > > I think, the next step in that direction would be to enable data > > checksums by default. They make sense in most setups, > > Well, that is exactly the point that needs some pro

Re: Enable data checksums by default

2019-03-22 Thread Andres Freund
On 2019-03-22 18:01:32 +0100, Tomas Vondra wrote: > On 3/22/19 5:41 PM, Andres Freund wrote: > > Hi, > > > > On 2019-03-22 17:32:10 +0100, Tomas Vondra wrote: > >> On 3/22/19 5:10 PM, Andres Freund wrote: > >>> IDK, being able to verify in some form that backups aren't corrupted on > >>> an IO lev

Re: Enable data checksums by default

2019-03-22 Thread Tomas Vondra
On 3/22/19 5:41 PM, Andres Freund wrote: > Hi, > > On 2019-03-22 17:32:10 +0100, Tomas Vondra wrote: >> On 3/22/19 5:10 PM, Andres Freund wrote: >>> IDK, being able to verify in some form that backups aren't corrupted on >>> an IO level is mighty nice. That often does allow to detect the issue >>>

Re: Enable data checksums by default

2019-03-22 Thread Andres Freund
Hi, On 2019-03-22 17:32:10 +0100, Tomas Vondra wrote: > On 3/22/19 5:10 PM, Andres Freund wrote: > > IDK, being able to verify in some form that backups aren't corrupted on > > an IO level is mighty nice. That often does allow to detect the issue > > while one still has older backups around. > >

Re: Enable data checksums by default

2019-03-22 Thread Tomas Vondra
On 3/22/19 5:10 PM, Andres Freund wrote: > Hi, > > On 2019-03-22 12:07:22 -0400, Tom Lane wrote: >> Christoph Berg writes: >>> I think, the next step in that direction would be to enable data >>> checksums by default. They make sense in most setups, >>

Re: Enable data checksums by default

2019-03-22 Thread Andres Freund
Hi, On 2019-03-22 12:07:22 -0400, Tom Lane wrote: > Christoph Berg writes: > > I think, the next step in that direction would be to enable data > > checksums by default. They make sense in most setups, > > Well, that is exactly the point that needs some proof, not just >

Re: Enable data checksums by default

2019-03-22 Thread Tom Lane
Christoph Berg writes: > I think, the next step in that direction would be to enable data > checksums by default. They make sense in most setups, Well, that is exactly the point that needs some proof, not just an unfounded assertion. IMO, the main value of checksums is that they all

Enable data checksums by default

2019-03-22 Thread Christoph Berg
k, the next step in that direction would be to enable data checksums by default. They make sense in most setups, and people who plan to run very performance-critical systems where checksums might be too much need to tune many knobs anyway, and can as well choose to disable them manually, instead