> On 9 Mar 2021, at 19:12, Bruce Momjian wrote:
Since this patch is de-facto rejected I'll mark it withdrawn in the CF app to
save on cfbot bandwidth.
> Do we support or document the ability to create a standby with checksums
> from a primary without it, and is that a better approach?
Michael B
On Mon, Feb 15, 2021 at 02:02:02PM +0100, Daniel Gustafsson wrote:
> > On 11 Feb 2021, at 14:10, Bruce Momjian wrote:
> > I don't think anyone has done anything wrong --- rather, it is what we
> > are _trying_ to do that is complex.
>
> Global state changes in a cluster are complicated, and are u
> On 11 Feb 2021, at 14:10, Bruce Momjian wrote:
>
> On Wed, Feb 10, 2021 at 01:26:18PM -0500, Bruce Momjian wrote:
>> On Wed, Feb 10, 2021 at 03:25:58PM +0100, Magnus Hagander wrote:
>>> A fairly large amount of this complexity comes out of the fact that it
>>> now supports restarting and tracks
On Wed, Feb 10, 2021 at 01:26:18PM -0500, Bruce Momjian wrote:
> On Wed, Feb 10, 2021 at 03:25:58PM +0100, Magnus Hagander wrote:
> > A fairly large amount of this complexity comes out of the fact that it
> > now supports restarting and tracks checksums on a per-table basis. We
> > skipped this in
On 10/02/2021 16:25, Magnus Hagander wrote:
On Tue, Feb 9, 2021 at 9:54 AM Heikki Linnakangas wrote:
(I may have said this before, but) My overall high-level impression of
this patch is that it's really cmmplex for a feature that you use maybe
once in the lifetime of a cluster. I'm happy to re
On Wed, Feb 10, 2021 at 03:25:58PM +0100, Magnus Hagander wrote:
> On Tue, Feb 9, 2021 at 9:54 AM Heikki Linnakangas wrote:
> >
> > (I may have said this before, but) My overall high-level impression of
> > this patch is that it's really cmmplex for a feature that you use maybe
> > once in the lif
On Tue, Feb 9, 2021 at 9:54 AM Heikki Linnakangas wrote:
>
> (I may have said this before, but) My overall high-level impression of
> this patch is that it's really cmmplex for a feature that you use maybe
> once in the lifetime of a cluster. I'm happy to review but I'm not
> planning to commit th
Hi,
Am Mittwoch, den 10.02.2021, 15:06 +0900 schrieb Michael Paquier:
> On Tue, Feb 09, 2021 at 10:54:50AM +0200, Heikki Linnakangas wrote:
> > (I may have said this before, but) My overall high-level impression of this
> > patch is that it's really cmmplex for a feature that you use maybe once in
On Tue, Feb 09, 2021 at 10:54:50AM +0200, Heikki Linnakangas wrote:
> (I may have said this before, but) My overall high-level impression of this
> patch is that it's really cmmplex for a feature that you use maybe once in
> the lifetime of a cluster. I'm happy to review but I'm not planning to
> c
(I may have said this before, but) My overall high-level impression of
this patch is that it's really cmmplex for a feature that you use maybe
once in the lifetime of a cluster. I'm happy to review but I'm not
planning to commit this myself. I don't object if some other committer
picks this up
Revisiting an issue we discussed earlier:
On 25/11/2020 15:20, Daniel Gustafsson wrote:
On 23 Nov 2020, at 18:36, Heikki Linnakangas wrote:
On 17/11/2020 10:56, Daniel Gustafsson wrote:
I've reworked this in the attached such that the enable_ and
disable_ functions merely call into the launch
On 22/01/2021 14:21, Heikki Linnakangas wrote:
On 22/01/2021 13:55, Heikki Linnakangas wrote:
I read through the latest patch,
v31-0001-Support-checksum-enable-disable-in-a-running-clu.patch. Some
comments below:
One more thing:
In SetRelationNumChecks(), you should use SearchSysCacheCopy1()
On 22/01/2021 13:55, Heikki Linnakangas wrote:
I read through the latest patch,
v31-0001-Support-checksum-enable-disable-in-a-running-clu.patch. Some
comments below:
One more thing:
In SetRelationNumChecks(), you should use SearchSysCacheCopy1() to get a
modifiable copy of the tuple. Otherwis
I read through the latest patch,
v31-0001-Support-checksum-enable-disable-in-a-running-clu.patch. Some
comments below:
On 19/01/2021 14:32, Daniel Gustafsson wrote:
+ /*
+* Hold interrupts for the duration of xlogging to avoid the state of
data
+* checksums changing duri
Hi,
On Fri, Jan 15, 2021 at 11:32:56AM +0100, Daniel Gustafsson wrote:
> The attached v30 adds the proposed optimizations in this thread as previously
> asked for, as well as some small cleanups to the procsignal calling codepath
> (replacing single call functions with just calling the function) a
On Fri, Jan 15, 2021 at 11:33 AM Daniel Gustafsson wrote:
>
> The attached v30 adds the proposed optimizations in this thread as previously
> asked for, as well as some small cleanups to the procsignal calling codepath
> (replacing single call functions with just calling the function) and some
> f
On Thu, Jan 7, 2021 at 3:05 PM Daniel Gustafsson wrote:
>
> > On 5 Jan 2021, at 18:19, Michael Banck wrote:
>
> >> diff --git a/doc/src/sgml/ref/initdb.sgml b/doc/src/sgml/ref/initdb.sgml
> >> index 385ac25150..e3b0048806 100644
> >> --- a/doc/src/sgml/ref/initdb.sgml
> >> +++ b/doc/src/sgml/ref/
Hi,
On Thu, Jan 07, 2021 at 03:05:44PM +0100, Daniel Gustafsson wrote:
> > On 5 Jan 2021, at 18:19, Michael Banck wrote:
> >> +
> >> + Data pages are not checksum protected by default, but this can
> >> optionally be
> >> + enabled for a cluster.
This would also have to be rewritten to c
> On 5 Jan 2021, at 18:19, Michael Banck wrote:
>> diff --git a/doc/src/sgml/ref/initdb.sgml b/doc/src/sgml/ref/initdb.sgml
>> index 385ac25150..e3b0048806 100644
>> --- a/doc/src/sgml/ref/initdb.sgml
>> +++ b/doc/src/sgml/ref/initdb.sgml
>> @@ -219,6 +219,7 @@ PostgreSQL documentation
>>
On Tue, Jan 05, 2021 at 09:29:31PM +0100, Michael Banck wrote:
> > @@ -4938,13 +4949,299 @@ GetMockAuthenticationNonce(void)
> > /*
> > - * Are checksums enabled for data pages?
> > + * DataChecksumsNeedWrite
> > + * Returns whether data checksums must be written or not
> > + *
> > + * Are
Hi,
On Tue, Jan 05, 2021 at 12:18:07AM +0100, Daniel Gustafsson wrote:
> Attached is a rebase of the patch on top of current HEAD.
>
> cheers ./daniel
Some more comments/questions:
> diff --git a/src/backend/access/heap/heapam.c
> b/src/backend/access/heap/heapam.c
> index 53e997cd55..06c001f8
Hi,
On Tue, Jan 05, 2021 at 12:18:07AM +0100, Daniel Gustafsson wrote:
> > On 3 Dec 2020, at 10:37, Daniel Gustafsson wrote:
> > I've also done some tweaks to the tests to make them more robust as well as
> > comment updates and general tidying up here and there.
>
> Attached is a rebase of the
On 25/11/2020 15:20, Daniel Gustafsson wrote:
On 23 Nov 2020, at 18:36, Heikki Linnakangas wrote:
What happens is if you crash between UpdateControlFile() and XlogChecksum()?
Good point, that would not get the cluster to a consistent state. The
XlogChecksum should be performed before controlf
On 17/11/2020 10:56, Daniel Gustafsson wrote:
On 5 Oct 2020, at 13:36, Heikki Linnakangas wrote:
2. The signaling between enable_data_checksums() and the launcher process looks
funny to me. The general idea seems to be that enable_data_checksums() just
starts the launcher process, and the laun
On Fri, Nov 13, 2020 at 12:22 PM Heikki Linnakangas wrote:
> On 12/11/2020 15:17, Daniel Gustafsson wrote:
> >> On 5 Oct 2020, at 14:14, Heikki Linnakangas wrote:
> >> I would expect the checksums worker to be automatically started at
> postmaster startup. Can we make that happen?
> >
> > A dyna
On 12/11/2020 15:17, Daniel Gustafsson wrote:
On 5 Oct 2020, at 14:14, Heikki Linnakangas wrote:
I would expect the checksums worker to be automatically started at postmaster
startup. Can we make that happen?
A dynamic background worker has to be registered from a regular backend, so
it's not
> On 5 Oct 2020, at 14:14, Heikki Linnakangas wrote:
>
> Replying to an older message in this thread:
>
>>> + /*
>>> + * If we reach this point with checksums in inprogress state, we notify
>>> + * the user that they need to manually restart the process to enable
>>> + * checksums. This is becau
On 05/10/2020 17:25, Álvaro Herrera wrote:
On 2020-Oct-05, Heikki Linnakangas wrote:
The code in sendFile() in basebackup.c seems suspicious in that regard. It
calls DataChecksumsNeedVerify() once before starting to read the file. Isn't
it possible for the checksums flag to change while it's re
On 2020-Oct-05, Heikki Linnakangas wrote:
> The code in sendFile() in basebackup.c seems suspicious in that regard. It
> calls DataChecksumsNeedVerify() once before starting to read the file. Isn't
> it possible for the checksums flag to change while it's reading the file and
> sending it to the c
Replying to an older message in this thread:
+ /*
+ * If we reach this point with checksums in inprogress state, we notify
+ * the user that they need to manually restart the process to enable
+ * checksums. This is because we cannot launch a dynamic background worker
+ * directly from here, it
I looked at patch v22, and I can see two main issues:
1. The one that Robert talked about earlier: A backend checks the local
"checksums" state. If it's 'off', it writes a page without checksums.
How do you guarantee that the local state doesn't change in between? The
implicit assumption seems
> On 24 Sep 2020, at 06:27, Michael Paquier wrote:
>
> On Wed, Sep 23, 2020 at 02:34:36PM +0200, Daniel Gustafsson wrote:
>> While in the patch I realized that the relationlist saved the relkind but the
>> code wasn't actually using it, so I've gone ahead and removed that with a lot
>> fewer pall
On Wed, Sep 23, 2020 at 02:34:36PM +0200, Daniel Gustafsson wrote:
> While in the patch I realized that the relationlist saved the relkind but the
> code wasn't actually using it, so I've gone ahead and removed that with a lot
> fewer palloc calls as a result. The attached v22 fixes that and the ab
+ * changed to "inprogress-off", the barrier for mvoving to "off" can be
moving
+ * When disabling checksums, data_checksums will be set of "inprogress-off"
set *to*
+ get_namespace_name(RelationGetNamespace(reln)),
RelationGetRelationName(reln),
I think this palloc()s a new copy of the n
> On 7 Sep 2020, at 09:17, Michael Paquier wrote:
> Daniel, could you look at that?
I believe this boils down to a timing issue, I've included a fix in the v21
patch attached to a previous mail upthread.
cheers ./daniel
On Wed, Sep 02, 2020 at 02:22:25PM +0200, Daniel Gustafsson wrote:
> I unfortunately haven't had time to read the READ ONLY patch so I can't
> comment
> on how these two patches do things in relation to each other.
>
> The main synchronization mechanisms are the use of the inprogress mode where
>
> On 28 Jul 2020, at 04:33, Justin Pryzby wrote:
>
> On Thu, Jun 25, 2020 at 11:43:00AM +0200, Daniel Gustafsson wrote:
>> The attached v19 fixes a few doc issues I had missed.
>
> + They can also be enabled or disabled at a later timne, either as an
> offline
> => time
Fixed.
> + * aw
> On 29 Jul 2020, at 19:58, Robert Haas wrote:
> Here are a bunch of comments based on a partial read-through of this
> patch.
Thanks a lot Robert and Justin for the reviews! With the commitfest wrap-up
imminent and being on vacation I will have a hard time responding properly
before the end of
On Mon, Jun 22, 2020 at 8:27 AM Daniel Gustafsson wrote:
> Attached is a new version of the online checksums patch which, I hope, address
> most of the concerns raised in previous reviews. There has been a fair amount
> of fiddling done, so below is a summary of what has been done.
Here are a bu
On Thu, Jun 25, 2020 at 11:43:00AM +0200, Daniel Gustafsson wrote:
> The attached v19 fixes a few doc issues I had missed.
+ They can also be enabled or disabled at a later timne, either as an offline
=> time
+* awaiting shutdown, but we can continue turning off checksums anyway
=> a wa
> On 26 Jun 2020, at 14:12, Robert Haas wrote:
>
> On Thu, Jun 25, 2020 at 5:43 AM Daniel Gustafsson wrote:
>> Sorry being a bit thick, can you elaborate which case you're thinking about?
>> CREATE TABLE sets the attribute according to the value of data_checksums, and
>> before enabling checksum
On Thu, Jun 25, 2020 at 5:43 AM Daniel Gustafsson wrote:
> Sorry being a bit thick, can you elaborate which case you're thinking about?
> CREATE TABLE sets the attribute according to the value of data_checksums, and
> before enabling checksums (and before changing data_checksums to inprogress)
> t
> On 22 Jun 2020, at 18:29, Robert Haas wrote:
>
> On Mon, Jun 22, 2020 at 8:27 AM Daniel Gustafsson wrote:
>> Restartability is implemented by keeping state in pg_class. I opted for a
>> bool
>> which is cleared as the first step of checksum enable, since it offers fewer
>> synchronization co
On Mon, Jun 22, 2020 at 8:27 AM Daniel Gustafsson wrote:
> Restartability is implemented by keeping state in pg_class. I opted for a
> bool
> which is cleared as the first step of checksum enable, since it offers fewer
> synchronization cornercases I think.
Unless you take AccessExclusiveLock o
> On 23 Jan 2020, at 18:23, Robert Haas wrote:
> ..That's probably easier and more efficient if it's just value
> in pg_class than if they have to go poking around in another catalog.
> So I am tentatively inclined to think that just putting it in pg_class
> makes more sense.
..which is what I d
On 4/1/20 11:30 AM, David Steele wrote:
On 1/18/20 6:18 PM, Daniel Gustafsson wrote:
Attached is a v16 rebased on top of current master which addresses the
above
commented points, and which I am basing the concurrency work on.
This patch no longer applies cleanly:
http://cfbot.cputube.org/
On 1/18/20 6:18 PM, Daniel Gustafsson wrote:
Attached is a v16 rebased on top of current master which addresses the above
commented points, and which I am basing the concurrency work on.
This patch no longer applies cleanly:
http://cfbot.cputube.org/patch_27_2260.log
The CF entry has been u
Hi,
On 2020-01-23 12:23:09 -0500, Robert Haas wrote:
> On Thu, Jan 23, 2020 at 6:19 AM Daniel Gustafsson wrote:
> > A bigger question is how to handle the offline capabilities. pg_checksums
> > can
> > enable or disable checksums in an offline cluster, which will put the
> > cluster
> > in a s
On Thu, Jan 23, 2020 at 6:19 AM Daniel Gustafsson wrote:
> A bigger question is how to handle the offline capabilities. pg_checksums can
> enable or disable checksums in an offline cluster, which will put the cluster
> in a state where the pg_control file and the catalog don't match at startup.
>
> On 22 Jan 2020, at 23:07, Robert Haas wrote:
>
> On Wed, Jan 22, 2020 at 3:28 PM Magnus Hagander wrote:
>>> I think the argument about adding catalog flags adding overhead is
>>> pretty much bogus. Fixed-width fields in catalogs are pretty cheap.
>>
>> If that's the general view, then yeah ou
On Wed, Jan 22, 2020 at 3:28 PM Magnus Hagander wrote:
> > I think the argument about adding catalog flags adding overhead is
> > pretty much bogus. Fixed-width fields in catalogs are pretty cheap.
>
> If that's the general view, then yeah our "cost calculations" were
> off. I guess I may have bee
On Wed, Jan 22, 2020 at 12:20 PM Robert Haas wrote:
>
> On Wed, Jan 22, 2020 at 2:50 PM Magnus Hagander wrote:
> > The reasoning that led us to *not* doing that is that it's a one-off
> > operation. That along with the fact that we hope to at some point be
> > able to change the default to chekcs
On Wed, Jan 22, 2020 at 2:50 PM Magnus Hagander wrote:
> The reasoning that led us to *not* doing that is that it's a one-off
> operation. That along with the fact that we hope to at some point be
> able to change the default to chekcsums on (and t wouldn't be
> necessary for the transition on->o
On Mon, Jan 20, 2020 at 12:14 PM Robert Haas wrote:
>
> On Sat, Jan 18, 2020 at 6:18 PM Daniel Gustafsson wrote:
> > Thanks again for reviewing (and working on the infrastructure required for
> > this
> > patch to begin with)! Regarding the persisting the progress; that would be
> > a
> > rea
On Sat, Jan 18, 2020 at 6:18 PM Daniel Gustafsson wrote:
> Thanks again for reviewing (and working on the infrastructure required for
> this
> patch to begin with)! Regarding the persisting the progress; that would be a
> really neat feature but I don't have any suggestion on how to do that safe
> On 3 Jan 2020, at 23:07, Robert Haas wrote:
>
> On Mon, Dec 16, 2019 at 10:16 AM Daniel Gustafsson wrote:
>> If reviewers think this version is nearing completion, then a v16 should
>> address the comment below, but as this version switches its underlying
>> infrastructure it seemed usefel for
On Mon, Dec 16, 2019 at 10:16 AM Daniel Gustafsson wrote:
> If reviewers think this version is nearing completion, then a v16 should
> address the comment below, but as this version switches its underlying
> infrastructure it seemed usefel for testing still.
I think this patch still needs a lot o
Hello
> Attached is a v15 of the online checksums patchset (minus 0005), rebased on
> top
> of your v3 ProcSignalBarrier patch rather than Andres' PoC GlobalBarrier
> patch.
> It does take the, perhaps, controversial approach of replacing the SAMPLE
> barrier with the CHECKSUM barrier. The cfbot
> On 5 Dec 2019, at 16:13, Robert Haas wrote:
>
> On Tue, Dec 3, 2019 at 6:41 PM Daniel Gustafsson wrote:
>> Attached is a rebased v14 patchset on top of maser. The Global Barriers
>> patch
>> is left as a prerequisite, but it will obviously be dropped, or be
>> significantly changed, once the
On Thu, Dec 5, 2019 at 10:28 AM Daniel Gustafsson wrote:
> I am currently reviewing your latest patchset, but need a bit more time.
Oh, great, thanks.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
> On 5 Dec 2019, at 16:13, Robert Haas wrote:
>
>> On Tue, Dec 3, 2019 at 6:41 PM Daniel Gustafsson wrote:
>> Attached is a rebased v14 patchset on top of maser. The Global Barriers
>> patch
>> is left as a prerequisite, but it will obviously be dropped, or be
>> significantly changed, once th
On Tue, Dec 3, 2019 at 6:41 PM Daniel Gustafsson wrote:
> Attached is a rebased v14 patchset on top of maser. The Global Barriers patch
> is left as a prerequisite, but it will obviously be dropped, or be
> significantly changed, once the work Robert is doing with ProcSignalBarrier
> lands.
Any
> On 1 Dec 2019, at 03:32, Michael Paquier wrote:
> The latest patch does not apply, could you send a rebase? Moved it to
> next CF, waiting on author.
Attached is a rebased v14 patchset on top of maser. The Global Barriers patch
is left as a prerequisite, but it will obviously be dropped, or
On Wed, Oct 02, 2019 at 08:59:27PM +0200, Magnus Hagander wrote:
> Much appreciated!
The latest patch does not apply, could you send a rebase? Moved it to
next CF, waiting on author.
--
Michael
signature.asc
Description: PGP signature
On Mon, Sep 30, 2019 at 6:11 PM Andres Freund wrote:
> Hi,
>
> On 2019-09-30 16:59:00 +0200, Magnus Hagander wrote:
> > On Mon, Sep 30, 2019 at 2:49 PM Tomas Vondra <
> tomas.von...@2ndquadrant.com>
> > wrote:
> > > IMHO the patch is ready to go - I think the global barrier solves the
> > > issue
Hi,
On 2019-09-30 16:59:00 +0200, Magnus Hagander wrote:
> On Mon, Sep 30, 2019 at 2:49 PM Tomas Vondra
> wrote:
> > IMHO the patch is ready to go - I think the global barrier solves the
> > issue in the previous version, and that's the only problem I'm aware of.
> > So +1 from me to go ahead and
On Mon, Sep 30, 2019 at 2:49 PM Tomas Vondra
wrote:
> On Mon, Sep 30, 2019 at 01:03:20PM +0200, Magnus Hagander wrote:
> >On Thu, Sep 26, 2019 at 9:48 PM Alvaro Herrera
> >wrote:
> >
> >> On 2019-Aug-26, Magnus Hagander wrote:
> >>
> >> > OK, let's try this again :)
> >> >
> >> > This is work ma
On Mon, Sep 30, 2019 at 04:57:41PM +0200, Magnus Hagander wrote:
>
>
> On Mon, Sep 30, 2019 at 4:53 PM Bruce Momjian wrote:
>
> On Mon, Sep 30, 2019 at 02:49:44PM +0200, Tomas Vondra wrote:
> > On Mon, Sep 30, 2019 at 01:03:20PM +0200, Magnus Hagander wrote:
> > > Other than bots, t
On Mon, Sep 30, 2019 at 4:53 PM Bruce Momjian wrote:
> On Mon, Sep 30, 2019 at 02:49:44PM +0200, Tomas Vondra wrote:
> > On Mon, Sep 30, 2019 at 01:03:20PM +0200, Magnus Hagander wrote:
> > > Other than bots, this patch doesn't seem to have attracted any
> reviewers
> > > > this time around. Per
On Mon, Sep 30, 2019 at 02:49:44PM +0200, Tomas Vondra wrote:
> On Mon, Sep 30, 2019 at 01:03:20PM +0200, Magnus Hagander wrote:
> > Other than bots, this patch doesn't seem to have attracted any reviewers
> > > this time around. Perhaps you need to bribe someone? (Maybe "how sad
> > > your commi
On Mon, Sep 30, 2019 at 01:03:20PM +0200, Magnus Hagander wrote:
On Thu, Sep 26, 2019 at 9:48 PM Alvaro Herrera
wrote:
On 2019-Aug-26, Magnus Hagander wrote:
> OK, let's try this again :)
>
> This is work mainly based in the first version of the online checksums
> patch, but based on top of A
On Thu, Sep 26, 2019 at 9:48 PM Alvaro Herrera
wrote:
> On 2019-Aug-26, Magnus Hagander wrote:
>
> > OK, let's try this again :)
> >
> > This is work mainly based in the first version of the online checksums
> > patch, but based on top of Andres WIP patchset for global barriers (
> >
> https://ww
On 2019-Aug-26, Magnus Hagander wrote:
> OK, let's try this again :)
>
> This is work mainly based in the first version of the online checksums
> patch, but based on top of Andres WIP patchset for global barriers (
> https://www.postgresql.org/message-id/20181030051643.elbxjww5jjgnjaxg%40alap3.an
OK, let's try this again :)
This is work mainly based in the first version of the online checksums
patch, but based on top of Andres WIP patchset for global barriers (
https://www.postgresql.org/message-id/20181030051643.elbxjww5jjgnjaxg%40alap3.anarazel.de
)
Andres patch has been enhanced with w
74 matches
Mail list logo
