Re: Complete data erasure

2020-04-11 Thread Tomas Vondra
On Sat, Apr 11, 2020 at 01:56:10PM -0400, Tom Lane wrote: Tomas Vondra writes: I don't think "commit is atomic" really implies "data should be released at commit". This is precisely what makes the feature extremely hard to implement, IMHO. Why wouldn't it be acceptable to do something like t

Re: Complete data erasure

2020-04-11 Thread Tom Lane
Tomas Vondra writes: > I don't think "commit is atomic" really implies "data should be released > at commit". This is precisely what makes the feature extremely hard to > implement, IMHO. > Why wouldn't it be acceptable to do something like this? > BEGIN; > ... > DROP TABLE x ERAS

Re: Complete data erasure

2020-04-11 Thread Tomas Vondra
On Fri, Apr 10, 2020 at 08:23:32AM +, asaba.takan...@fujitsu.com wrote: Hello, I was off the point. I want to organize the discussion and suggest feature design. There are two opinions. 1. COMMIT should not take a long time because errors are more likely to occur. I don't think it's a mat

RE: Complete data erasure

2020-04-10 Thread asaba.takan...@fujitsu.com
Hello, I was off the point. I want to organize the discussion and suggest feature design. There are two opinions. 1. COMMIT should not take a long time because errors are more likely to occur. 2. The data area should be released when COMMIT is completed because COMMIT has to be an atomic action.

RE: Complete data erasure

2020-03-17 Thread asaba.takan...@fujitsu.com
Hello Tom, From: asaba.takan...@fujitsu.com > Hello Tom, > > From: Tom Lane > > Tomas Vondra writes: > > > I think it depends how exactly it's implemented. As Tom pointed out in > > > his message [1], we can't do the erasure itself in the post-commit is > > > not being able to handle errors. B

RE: Complete data erasure

2020-02-20 Thread asaba.takan...@fujitsu.com
Hello Tom, From: Tom Lane > Tomas Vondra writes: > > I think it depends how exactly it's implemented. As Tom pointed out in > > his message [1], we can't do the erasure itself in the post-commit is > > not being able to handle errors. But if the files are renamed durably, > > and the erasure hap

RE: Complete data erasure

2020-02-20 Thread asaba.takan...@fujitsu.com
Greetings, From: asaba.takan...@fujitsu.com > Hello Stephen, > > From: Stephen Frost > > I disagree- it's a feature that's been asked for multiple times and does > > have value in some situations. > > I'm rethinking the need for this feature although I think that it improves the > security. >

RE: Complete data erasure

2020-02-09 Thread asaba.takan...@fujitsu.com
Hello Stephen, From: Stephen Frost > I disagree- it's a feature that's been asked for multiple times and does > have value in some situations. I'm rethinking the need for this feature although I think that it improves the security. You said that this feature has value in some situations. Could

Re: Complete data erasure

2020-02-05 Thread Masahiko Sawada
On Tue, 4 Feb 2020 at 09:53, tsunakawa.ta...@fujitsu.com wrote: > > From: Tomas Vondra > > That's not really what I meant - let me explain. When I said DROP TABLE > > should do everything as usual, that includes catalog changes. I.e. after > > the commit there would not be any remaining entries i

RE: Complete data erasure

2020-02-04 Thread tsunakawa.ta...@fujitsu.com
From: Tom Lane > Up to now, we've sort of looked the other way with respect to failures > of file unlinks post-commit, reasoning that the worst that will happen > is disk space leakage from no-longer-referenced files that we failed to > unlink. (Which is bad, certainly, but not catastrophic; it's

Re: Complete data erasure

2020-02-04 Thread Tom Lane
Tomas Vondra writes: > I think it depends how exactly it's implemented. As Tom pointed out in > his message [1], we can't do the erasure itself in the post-commit is > not being able to handle errors. But if the files are renamed durably, > and the erasure happens in a separate process, that could

Re: Complete data erasure

2020-02-04 Thread Tomas Vondra
On Tue, Feb 04, 2020 at 12:53:44AM +, tsunakawa.ta...@fujitsu.com wrote: From: Tomas Vondra That's not really what I meant - let me explain. When I said DROP TABLE should do everything as usual, that includes catalog changes. I.e. after the commit there would not be any remaining entries in

RE: Complete data erasure

2020-02-03 Thread imai.yoshik...@fujitsu.com
>From tsunakawa.ta...@fujitsu.com > What is concerned about is that the need to erase and delete the data file > would be forgotten if the server crashes during step > 3. If so, postmaster can do the job at startup, just like it deletes > temporary files (although it delays the startup.) I sus

RE: Complete data erasure

2020-02-03 Thread tsunakawa.ta...@fujitsu.com
From: Tomas Vondra > That's not really what I meant - let me explain. When I said DROP TABLE > should do everything as usual, that includes catalog changes. I.e. after > the commit there would not be any remaining entries in system catalogs > or anything like that. > > The only thing we'd do diff

Re: Complete data erasure

2020-02-03 Thread Tomas Vondra
On Mon, Feb 03, 2020 at 09:07:09AM -0500, Stephen Frost wrote: Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: On Tue, Jan 28, 2020 at 02:34:07PM -0500, Stephen Frost wrote: >We certainly can't run external commands during transaction COMMIT, so >this can't be part of a regular

Re: Complete data erasure

2020-02-03 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > On Tue, Jan 28, 2020 at 02:34:07PM -0500, Stephen Frost wrote: > >We certainly can't run external commands during transaction COMMIT, so > >this can't be part of a regular DROP TABLE. > > IMO the best solution would be that the DRO

Re: Complete data erasure

2020-01-28 Thread Tomas Vondra
On Tue, Jan 28, 2020 at 02:34:07PM -0500, Stephen Frost wrote: Greetings, * asaba.takan...@fujitsu.com (asaba.takan...@fujitsu.com) wrote: From: Stephen Frost > * asaba.takan...@fujitsu.com (asaba.takan...@fujitsu.com) wrote: > > This feature erases data area just before it is returned to the

Re: Complete data erasure

2020-01-28 Thread Stephen Frost
Greetings, * asaba.takan...@fujitsu.com (asaba.takan...@fujitsu.com) wrote: > From: Stephen Frost > > * asaba.takan...@fujitsu.com (asaba.takan...@fujitsu.com) wrote: > > > This feature erases data area just before it is returned to the OS > > > (“erase” > > means that overwrite data area to hid

RE: Complete data erasure

2020-01-21 Thread asaba.takan...@fujitsu.com
Hello Stephen, Thank you for comment. From: Stephen Frost > Greetings, > > * asaba.takan...@fujitsu.com (asaba.takan...@fujitsu.com) wrote: > > This feature erases data area just before it is returned to the OS (“erase” > means that overwrite data area to hide its contents here) > > because the

Re: Complete data erasure

2020-01-17 Thread Stephen Frost
Greetings, * asaba.takan...@fujitsu.com (asaba.takan...@fujitsu.com) wrote: > This feature erases data area just before it is returned to the OS (“erase” > means that overwrite data area to hide its contents here) > because there is a risk that the data will be restored by attackers if it is >

RE: Complete data erasure

2020-01-17 Thread asaba.takan...@fujitsu.com
Hello, Horiguchi-san Thank you for comment. At Wed, 15 Jan 2020 03:46 +, "Kyotaro Horiguchi " wrote in > shred(1) or wipe(1) doesn't seem to contribute to the objective on > journaled or copy-on-write file systems. I'm not sure, but maybe the > same can be true for read-modify-write devices

Re: Complete data erasure

2020-01-15 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Tomas Vondra writes: > > But let's assume it makes sense - is this really the right solution? I > > think what I'd prefer is encryption + rotation of the keys. Which should > > work properly even on COW filesystems, the performance impact is kin

Re: Complete data erasure

2020-01-15 Thread Tom Lane
Tomas Vondra writes: > But let's assume it makes sense - is this really the right solution? I > think what I'd prefer is encryption + rotation of the keys. Which should > work properly even on COW filesystems, the performance impact is kinda > low and amortized etc. Of course, we're discussing bui

Re: Complete data erasure

2020-01-15 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > On Wed, Jan 15, 2020 at 10:23:22AM -0500, Stephen Frost wrote: > >I disagree entirely. If the operating system and hardware level provide > >a way for this to work, which is actually rather common when you > >consider that ext4 is

Re: Complete data erasure

2020-01-15 Thread Tomas Vondra
On Wed, Jan 15, 2020 at 10:23:22AM -0500, Stephen Frost wrote: Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: "asaba.takan...@fujitsu.com" writes: > I want to add the feature to erase data so that it cannot be restored > because it prevents attackers from stealing data from released data ar

Re: Complete data erasure

2020-01-15 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > "asaba.takan...@fujitsu.com" writes: > > I want to add the feature to erase data so that it cannot be restored > > because it prevents attackers from stealing data from released data area. > > I think this is fairly pointless, unfortunately.

Re: Complete data erasure

2020-01-14 Thread Tom Lane
"asaba.takan...@fujitsu.com" writes: > I want to add the feature to erase data so that it cannot be restored > because it prevents attackers from stealing data from released data area. I think this is fairly pointless, unfortunately. Dropping or truncating tables is as much as we can do without

Re: Complete data erasure

2020-01-14 Thread Kyotaro Horiguchi
Hello, Asaba-san. At Wed, 15 Jan 2020 01:31:44 +, "asaba.takan...@fujitsu.com" wrote in > Hello hackers, > > I want to add the feature to erase data so that it cannot be restored > because it prevents attackers from stealing data from released data area. > > - Background > International