Florian G. Pflug wrote:
August Zajonc wrote:
I'm confused about this.
As long as we assert the rule that the file name can't change on the
move, then after commit the file can be in only one of two places.
The name of the file is known (ie, pg_class). The directories are
known. What needs to be carried forwarded past a checkpoint? We don't
even look at WAL, so checkpoints are irrelevant it seems
If there is a crash just after commit and before the move, no harm.
You just move on startup. If the move fails, no harm, you can emit
warning and open in /pending (or simply error, even easier).
If you're going to open the file from /pending, whats the point of moving
it in the first place?
Allow time for someone to sort out the disk situation without impacting
things. Preserves the concept that after COMMIT a file exists on disk
that is accessible, which is what people I think expect.
The idea would have to be that you move on commit (Or on COMMIT-record
replay, in case of a crash), and then, after recovering the whole wal,
you could remove leftover files in /pending.
I think so. What I was thinking was
Limit to moves from spclocation/file.new to spclocation/file. So given a
pg_class filename the datafile can only have two possible names.
relfilenode or relfilenode.new
you commit, then move.
If crash occurs before commit, you leak a .new table.
If move fails after commit you emit warning. The commit is still valid,
because fopen can fall back to .new. The data is still there.
On crash recovery move .new files that show up in pg_class to their
proper name if the disk has been fixed and move can succeed. For .new
files that don't exist in pg_class after log replay, delete them.
Fallback open.
fopen relfilenode,
if ENOENT fopen relfilenode.new
if ENOENT error
elseif emit warning
So, what are you going to do if the move fails? You cannot roll back, and
you cannot update the CLOG (because than others would see your new table,
but no datafile). The only option is to PANIC. This will lead to a server
restart, WAL recovery, and probably another PANIC once the COMMIT-record
is replayed (Since the move probably still won't be possible).
That was the idea of the fallback generally, avoid this issue. You never
rollback. If datafile is not where expected, it can only be one other
place.
It might be even worse - I'm not sure that a rename is an atomic
operation
on most filesystems. If it's not, then you might end up with two files if
power fails *just* as you rename, or, worse with no file at all. Even
a slight
possibility of the second case seems unacceptable - I means loosing
a committed transaction.
Yes, atomic renames are an assumption.
I agree that we should eventually find a way to guarantee either no file
leakage, or at least an upper bound on the amount of wasted space. But
doing so at the cost of PANICing if the move fails seems like a bad
tradeoff...
Agreed...
greetings, Florian Pflug
---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster
