Re: Creation of an empty table is not fsync'd at checkpoint

2023-07-04 Thread Heikki Linnakangas
On 03/07/2023 15:59, Daniel Gustafsson wrote: This patch required a trivial rebase after conflicting with bfcf1b3480 so I've attached a v2 to get the CFBot to run this. Thank you! Pushed to all supported branches. (Without forgetting the new REL_16_STABLE :-D ). -- Heikki Linnakangas Neon (h

Re: Creation of an empty table is not fsync'd at checkpoint

2023-07-03 Thread Daniel Gustafsson
This patch required a trivial rebase after conflicting with bfcf1b3480 so I've attached a v2 to get the CFBot to run this. -- Daniel Gustafsson v2-0001-Ensure-that-creation-of-an-empty-relfile-is-fsync.patch Description: Binary data

Re: Creation of an empty table is not fsync'd at checkpoint

2022-01-27 Thread Thomas Munro
On Fri, Jan 28, 2022 at 11:39 AM Heikki Linnakangas wrote: > Hmm, if a relation is dropped, we use plain unlink() to delete it (at > the next checkpoint). Should we use durable_unlink() there, or otherwise > arrange to fsync() the parent directory? Hm. I think the latter might be a good idea

Re: Creation of an empty table is not fsync'd at checkpoint

2022-01-27 Thread Thomas Munro
On Fri, Jan 28, 2022 at 12:36 PM Andres Freund wrote: > On 2022-01-28 00:39:22 +0200, Heikki Linnakangas wrote: > > On 28/01/2022 00:11, Thomas Munro wrote: > > > ... but we still never synchronize "base/5". According to our > > > project's reading of the POSIX tea leaves we should be doing that

Re: Creation of an empty table is not fsync'd at checkpoint

2022-01-27 Thread Andres Freund
Hi, On 2022-01-28 00:39:22 +0200, Heikki Linnakangas wrote: > On 28/01/2022 00:11, Thomas Munro wrote: > > ... but we still never synchronize "base/5". According to our > > project's reading of the POSIX tea leaves we should be doing that to > > nail down the directory entry. > > Really? 'base/5

Re: Creation of an empty table is not fsync'd at checkpoint

2022-01-27 Thread Heikki Linnakangas
On 28/01/2022 00:11, Thomas Munro wrote: On Fri, Jan 28, 2022 at 8:12 AM Thomas Munro wrote: On Fri, Jan 28, 2022 at 6:55 AM Heikki Linnakangas wrote: I think the simplest fix is to call register_dirty_segment() from mdcreate(). As in the attached. Thoughts? +1 [Testing] Erm, so now I se

Re: Creation of an empty table is not fsync'd at checkpoint

2022-01-27 Thread Thomas Munro
On Fri, Jan 28, 2022 at 8:12 AM Thomas Munro wrote: > On Fri, Jan 28, 2022 at 6:55 AM Heikki Linnakangas wrote: > > I think the simplest fix is to call register_dirty_segment() from > > mdcreate(). As in the attached. Thoughts? > > +1 [Testing] Erm, so now I see my new table in checkpoint's act

Re: Creation of an empty table is not fsync'd at checkpoint

2022-01-27 Thread Thomas Munro
On Fri, Jan 28, 2022 at 8:17 AM Andres Freund wrote: > On 2022-01-28 08:01:01 +1300, Thomas Munro wrote: > > It might be possible to avoid that on xfs or pretty much any other > > file system. I wasn't following this closely, but even with ext4's > > recent fast commit changes, its fsync implemen

Re: Creation of an empty table is not fsync'd at checkpoint

2022-01-27 Thread Andres Freund
Hi, On 2022-01-28 08:01:01 +1300, Thomas Munro wrote: > On Fri, Jan 28, 2022 at 7:28 AM Andres Freund wrote: > > On 2022-01-27 19:55:45 +0200, Heikki Linnakangas wrote: > > > I was not able to reproduce this without the tablespace on a different > > > virtual disk, I presume because ext4 orders t

Re: Creation of an empty table is not fsync'd at checkpoint

2022-01-27 Thread Thomas Munro
On Fri, Jan 28, 2022 at 6:55 AM Heikki Linnakangas wrote: > I think the simplest fix is to call register_dirty_segment() from > mdcreate(). As in the attached. Thoughts? +1

Re: Creation of an empty table is not fsync'd at checkpoint

2022-01-27 Thread Thomas Munro
On Fri, Jan 28, 2022 at 7:28 AM Andres Freund wrote: > On 2022-01-27 19:55:45 +0200, Heikki Linnakangas wrote: > > I was not able to reproduce this without the tablespace on a different > > virtual disk, I presume because ext4 orders the writes so that the > > checkpoint implicitly always flushes

Re: Creation of an empty table is not fsync'd at checkpoint

2022-01-27 Thread Andres Freund
Hi, On 2022-01-27 19:55:45 +0200, Heikki Linnakangas wrote: > If you create an empty table, it is not fsync'd. As soon as you insert a row > to it, register_dirty_segment() gets called, and after that, the next > checkpoint will fsync it. But before that, the creation itself is never > fsync'd. Th

Creation of an empty table is not fsync'd at checkpoint

2022-01-27 Thread Heikki Linnakangas
If you create an empty table, it is not fsync'd. As soon as you insert a row to it, register_dirty_segment() gets called, and after that, the next checkpoint will fsync it. But before that, the creation itself is never fsync'd. That's obviously not great. The lack of an fsync is a bit hard to