Greetings,
* Bruce Momjian (br...@momjian.us) wrote:
> On Thu, Jul 25, 2019 at 03:55:01PM -0400, Stephen Frost wrote:
> > * Bruce Momjian (br...@momjian.us) wrote:
> > > On Thu, Jul 25, 2019 at 03:41:05PM -0400, Stephen Frost wrote:
> > > > * Bruce Momjian (br...@momjian.us) wrote:
> > > > > After talking to Joe Conway, I just want to mention that if we decide
> > > > > that the LSN is unique among heap and index, or among heap or index,
> > > > > we
> > > > > will need to make sure future WAL records retain this uniqueness.
> > > >
> > > > One thing comes to mind regarding this and I'll admit that I don't quite
> > > > remember exactly off-hand but I also don't want to not mention it now
> > > > and forget to later.
> > > >
> > > > What about pg_upgrade?
> > >
> > > So, we don't carry WAL from the old cluster to the new cluster, so if
> > > the WAL is changed and had duplicates, it would only be new WAL records.
> >
> > Right, we don't carry it forward- but what I couldn't remember is if
> > start from more-or-less LSN 0 or if pg_upgrade will arrange it such that
> > the new major version will start from LSN-of-old+1 (or whatever). Seems
> > like it'd *have* to be the latter, but just thought of it and wanted to
> > make sure.
>
> pg_upgrade uses pg_resetwal -l to set the next WAL segment file based on
> the value in the old cluster:
>
> /* now reset the wal archives in the new cluster */
> prep_status("Resetting WAL archives");
> exec_prog(UTILITY_LOG_FILE, NULL, true, true,
> /* use timeline 1 to match controldata and no WAL history file */
> --> "\"%s/pg_resetwal\" -l 00000001%s \"%s\"", new_cluster.bindir,
> old_cluster.controldata.nextxlogfile + 8,
> new_cluster.pgdata);Ah, right, ok, we reset the timeline but not the LSN, ok. > > > pg_upgrade seems immune to must of this, and that is by design. > > > However, I am hesitant to change the heap/index page format for > > > encryption because if we add fields, old pages might not fit as > > > encrypted pages, and then you have to move rows around, and things > > > become _much_ more complicated. > > > > Yeah, I'm afraid we are going to have a hard time making this work > > without changing the page format for encrypted.. I don't know if that's > > actually a *huge* issue like we've considered it to be in the past or > > not, as making someone rewrite just the few sensitive tables in their > > environment might not be that bad, and there's also logical replication > > today.. > > It is hard to do that while the server is offline. I don't see any reason to assume we must only support encrypting individual tables when the server is offline, or that we have to support any option which involves the server being offline when it comes to doing encryption. I'm not against supporting a "shut down the server and then encrypt everything and then start it up" option, but I don't see any particularly good reason to explicitly design the system with that use-case in mind. There seems to be a strong thrust on this thread to assume that a database MUST go from ALL DECRYPTED to ALL ENCRYPTED in one shot (and therefore we have to shut down the server to do it), but I don't get why that's the case, particularly if we support any kind of mixed setup where there's some data that's encrypted and some that isn't, and since we're talking about using different keys for different things, it seems to me that we almost certainly should be able to easily say "well, there's no key for this, so just don't go through the decrypt/encrypt routines". I can see an argument for why we might need to go through a restart and possibly use some off-line tool when a user decides they want, say, the WAL, or CLOG, or the other control files, to be encrypted, or basic encryption capability to be set up for the cluster (like generating the master key and storing it or making some changes to the control file to indicate that some things in this cluster has encrypted bits), but saying we must have the server offline to support encrypted tables that have a different page format is a bit like saying we need to take the server offline to add a new tableam (like zheap) and that we have to use some off-line utility while the server is down to convert a given table from heapam to zheap, isn't it? Thanks, Stephen
signature.asc
Description: PGP signature
