Re: [RFC v2 06/83] Add inode get/read methods.

On Sun, Apr 22, 2018 at 11:12 PM, Darrick J. Wong wrote: > [haaa, I finally found time to read more of these] > > On Sat, Mar 10, 2018 at 10:17:47AM -0800, Andiry Xu wrote: >> From: Andiry Xu >> >> These routines are incomplete and currently only support reserved ino

Re: [RFC v2 83/83] Sysfs support.

On Thu, Mar 22, 2018 at 8:00 AM, David Sterba wrote: > On Sat, Mar 10, 2018 at 10:19:04AM -0800, Andiry Xu wrote: >> From: Andiry Xu >> >> Sysfs support allows user to get/post information of running NOVA instance. >> After mount, NOVA creates four entries under proc

Re: [RFC v2 01/83] Introduction and documentation of NOVA filesystem.

Thanks for all the comments. On Mon, Mar 19, 2018 at 1:43 PM, Randy Dunlap wrote: > On 03/10/2018 10:17 AM, Andiry Xu wrote: >> From: Andiry Xu >> >> NOVA is a log-structured file system tailored for byte-addressable >> non-volatile memories. >> It was

Re: [RFC v2 05/83] Add NOVA filesystem definitions and useful helper routines.

On Mon, Mar 19, 2018 at 1:30 PM, Eric Biggers wrote: > On Mon, Mar 19, 2018 at 12:39:55PM -0700, Andiry Xu wrote: >> On Sun, Mar 11, 2018 at 12:22 PM, Eric Biggers wrote: >> > On Sun, Mar 11, 2018 at 02:00:13PM +0200, Nikolay Borisov wrote: >> >> [Adding He

Re: [RFC v2 05/83] Add NOVA filesystem definitions and useful helper routines.

On Sun, Mar 11, 2018 at 12:22 PM, Eric Biggers wrote: > On Sun, Mar 11, 2018 at 02:00:13PM +0200, Nikolay Borisov wrote: >> [Adding Herbert Xu to CC since he is the maintainer of the crypto subsys >> maintainer] >> >> On 10.03.2018 20:17, Andiry Xu wrote: >

Re: [RFC v2 03/83] Add super.h.

On Thu, Mar 15, 2018 at 7:59 PM, Theodore Y. Ts'o wrote: > On Thu, Mar 15, 2018 at 09:38:29PM +0100, Arnd Bergmann wrote: >> >> You could also have a resolution of less than a nanosecond. Note >> that today, the file time stamps generated by the kernel are in >> jiffies resolution, so at best one

Re: [RFC v2 03/83] Add super.h.

On Thu, Mar 15, 2018 at 2:05 AM, Arnd Bergmann wrote: > On Thu, Mar 15, 2018 at 7:11 AM, Andiry Xu wrote: >> On Wed, Mar 14, 2018 at 9:54 PM, Darrick J. Wong >> wrote: >>> On Sat, Mar 10, 2018 at 10:17:44AM -0800, Andiry Xu wrote: > >>>> + /* s_mtime a

Re: [RFC v2 04/83] NOVA inode definition.

On Wed, Mar 14, 2018 at 10:06 PM, Darrick J. Wong wrote: > On Sat, Mar 10, 2018 at 10:17:45AM -0800, Andiry Xu wrote: >> From: Andiry Xu >> >> inode.h defines the non-volatile and volatile NOVA inode data structures. >> >> The non-volatile NOVA inode (nova_i

Re: [RFC v2 03/83] Add super.h.

On Wed, Mar 14, 2018 at 9:54 PM, Darrick J. Wong wrote: > On Sat, Mar 10, 2018 at 10:17:44AM -0800, Andiry Xu wrote: >> From: Andiry Xu >> >> This header file defines NOVA persistent and volatile superblock >> data structures. >> >> It also defines NOVA

Re: [RFC v2 83/83] Sysfs support.

On Wed, Mar 14, 2018 at 5:33 PM, Randy Dunlap wrote: > On 03/10/2018 10:19 AM, Andiry Xu wrote: >> Sysfs support allows user to get/post information of running NOVA instance. >> After mount, NOVA creates four entries under proc directory >> /proc/fs/nova/pmem#/: >>

Re: [RFC v2 05/83] Add NOVA filesystem definitions and useful helper routines.

On Sun, Mar 11, 2018 at 12:22 PM, Eric Biggers wrote: > On Sun, Mar 11, 2018 at 02:00:13PM +0200, Nikolay Borisov wrote: >> [Adding Herbert Xu to CC since he is the maintainer of the crypto subsys >> maintainer] >> >> On 10.03.2018 20:17, Andiry Xu wrote: >

Re: [RFC v2 09/83] Add Kconfig and Makefile

On Sun, Mar 11, 2018 at 5:15 AM, Nikolay Borisov wrote: > > > On 10.03.2018 20:17, Andiry Xu wrote: >> From: Andiry Xu >> >> Signed-off-by: Andiry Xu >> --- >> fs/Kconfig | 2 ++ >> fs/Makefile | 1 + >> fs/nova/Kconfig | 15

Re: [RFC v2 14/83] Add range node kmem cache.

On Sun, Mar 11, 2018 at 4:55 AM, Nikolay Borisov wrote: > > > On 10.03.2018 20:17, Andiry Xu wrote: >> From: Andiry Xu >> >> Range node specifies a range of [start, end]. and is managed by a red-black >> tree. >> NOVA uses range node to manage NVM alloca

Re: [RFC v2 16/83] Initialize block map and free lists in nova_init().

On Sun, Mar 11, 2018 at 5:12 AM, Nikolay Borisov wrote: > > > On 10.03.2018 20:17, Andiry Xu wrote: >> From: Andiry Xu >> >> NOVA divides the pmem range equally among per-CPU free lists, >> and format the red-black trees by inserting the initial free rang

Re: [RFC v2 00/83] NOVA: a new file system for persistent memory

On Sat, Mar 10, 2018 at 6:14 PM, Theodore Y. Ts'o wrote: > FYI, your patch set doesn't even compile for me without these fixups. > I'm not sure why you were trying to declare inline functions in a > header file without the function body? > Thanks for catching this. I will fix it in the next versi

[RFC v2 04/83] NOVA inode definition.

From: Andiry Xu inode.h defines the non-volatile and volatile NOVA inode data structures. The non-volatile NOVA inode (nova_inode) is aligned to 128 bytes and contains file/directory metadata information. The most important fields are log_head and log_tail. log_head points to the start of the

[RFC v2 03/83] Add super.h.

From: Andiry Xu This header file defines NOVA persistent and volatile superblock data structures. It also defines NOVA block layout: Page 0: Superblock Page 1: Reserved inodes Page 2 - 15: Reserved Page 16 - 31: Inode table pointers Page 32 - 47: Journal address pointers Page 48 - 63: Reserved

[RFC v2 05/83] Add NOVA filesystem definitions and useful helper routines.

From: Andiry Xu NOVA stores offset rather than absolute addresses in pmem. nova_get_block() and nova_get_addr_off() provide transitions between these two kinds of addresses. Signed-off-by: Andiry Xu --- fs/nova/nova.h | 299 + 1 file

[RFC v2 06/83] Add inode get/read methods.

From: Andiry Xu These routines are incomplete and currently only support reserved inodes, whose addresses are fixed. This is necessary for fill_super to work. File/dir operations are left NULL. Signed-off-by: Andiry Xu --- fs/nova/inode.c | 176

[RFC v2 02/83] Add nova_def.h.

From: Andiry Xu This file defines NOVA filesystem macros and routines to persist updates by using Intel persistent memory instruction CLWB or clflush. Signed-off-by: Andiry Xu --- fs/nova/nova_def.h | 128 + 1 file changed, 128 insertions

[RFC v2 08/83] NOVA superblock operations.

From: Andiry Xu This is the entry point for NOVA filesystem mount and umount. NOVA works on DAX devices. During initialization it gets the device information, such as physical/virtual addresses and device size. It does not access the DAX device during runtime. During initialization NOVA also

[RFC v2 09/83] Add Kconfig and Makefile

From: Andiry Xu Signed-off-by: Andiry Xu --- fs/Kconfig | 2 ++ fs/Makefile | 1 + fs/nova/Kconfig | 15 +++ fs/nova/Makefile | 7 +++ 4 files changed, 25 insertions(+) create mode 100644 fs/nova/Kconfig create mode 100644 fs/nova/Makefile diff --git a/fs

[RFC v2 15/83] Add free list data structure.

From: Andiry Xu Free list is the data structure that NOVA uses to manage free pmem blocks. Each CPU has its own free list to avoid contention. Free list manages free pmem blocks (represented in range node) with red-black tree. Signed-off-by: Andiry Xu --- fs/nova/Makefile | 2 +- fs/nova

[RFC v2 07/83] Initialize inode_info and rebuild inode information in nova_iget().

From: Andiry Xu Incomplete nova_rebuild_inode() implemenation. nova_rebuild_inode() will go through the inode log and rebuild radix tree and metadata. Leave for later patches. Signed-off-by: Andiry Xu --- fs/nova/bbuild.c | 53 + fs/nova

[RFC v2 11/83] Add timing and I/O statistics for performance analysis and profiling.

From: Andiry Xu Signed-off-by: Andiry Xu --- fs/nova/Makefile | 2 +- fs/nova/nova.h | 12 +++ fs/nova/stats.c | 263 +++ fs/nova/stats.h | 178 + fs/nova/super.c | 6 ++ 5 files changed, 460

[RFC v2 12/83] Add timing for mount and init.

From: Andiry Xu Signed-off-by: Andiry Xu --- fs/nova/super.c | 25 ++--- 1 file changed, 22 insertions(+), 3 deletions(-) diff --git a/fs/nova/super.c b/fs/nova/super.c index 9295d23..3efb560 100644 --- a/fs/nova/super.c +++ b/fs/nova/super.c @@ -347,6 +347,9 @@ static

[RFC v2 14/83] Add range node kmem cache.

From: Andiry Xu Range node specifies a range of [start, end]. and is managed by a red-black tree. NOVA uses range node to manage NVM allocator and inodes being used. Signed-off-by: Andiry Xu --- fs/nova/nova.h | 8 fs/nova/super.c | 45

[RFC v2 16/83] Initialize block map and free lists in nova_init().

From: Andiry Xu NOVA divides the pmem range equally among per-CPU free lists, and format the red-black trees by inserting the initial free range. Signed-off-by: Andiry Xu --- fs/nova/balloc.c | 161 +++ fs/nova/balloc.h | 13 - fs/nova

[RFC v2 17/83] Add statfs support.

From: Andiry Xu Signed-off-by: Andiry Xu --- fs/nova/balloc.c | 18 ++ fs/nova/balloc.h | 1 + fs/nova/super.c | 19 +++ 3 files changed, 38 insertions(+) diff --git a/fs/nova/balloc.c b/fs/nova/balloc.c index cb627db..0742fe0 100644 --- a/fs/nova/balloc.c

[RFC v2 13/83] Add remount_fs and show_options methods.

From: Andiry Xu Signed-off-by: Andiry Xu --- fs/nova/super.c | 55 +++ 1 file changed, 55 insertions(+) diff --git a/fs/nova/super.c b/fs/nova/super.c index 3efb560..f41cc04 100644 --- a/fs/nova/super.c +++ b/fs/nova/super.c @@ -617,6

[RFC v2 19/83] Add pmem block free routines.

From: Andiry Xu NOVA allocates/frees log pages and data pages in the same way. For block free, NOVA first gets the corresponding free list by checking the block number, and then inserts the freed range in the red-black tree. NOVA always merge adjacent free ranges if possible. Signed-off-by

[RFC v2 20/83] Pmem block allocation routines.

From: Andiry Xu Upon a allocation request, NOVA first try the free list on current CPU. If there are not enough blocks to allocate, NOVA will go to the free list with the most free blocks. Caller can specify allocation direction: from low address or from high address. Signed-off-by: Andiry Xu

[RFC v2 18/83] Add freelist statistics printing.

From: Andiry Xu Signed-off-by: Andiry Xu --- fs/nova/nova.h | 1 + fs/nova/stats.c | 103 2 files changed, 104 insertions(+) diff --git a/fs/nova/nova.h b/fs/nova/nova.h index c4abdd8..404e133 100644 --- a/fs/nova/nova.h +++ b/fs

[RFC v2 23/83] Save allocator to pmem in put_super.

From: Andiry Xu We allocate log pages and append free range node to the log of the reserved blocknode inode. We can recover the allocator status by reading the log upon normal recovery. Signed-off-by: Andiry Xu --- fs/nova/bbuild.c | 114

[RFC v2 22/83] Inode log pages allocation and reclaimation.

From: Andiry Xu NOVA allocates one log page for each new inode. When the log is full, NOVA allocates new log pages, extends the log by either doubling the log size or increasing by fixed length, depends on log size. Signed-off-by: Andiry Xu --- fs/nova/Makefile | 2 +- fs/nova/log.c

[RFC v2 21/83] Add log structure.

From: Andiry Xu NOVA log is a singly linked list of 4KB pmem pages. Each log page consists of two parts: 4064 bytes for log entries, and 32 bytes for page tail structure. Page tail contains metadata about the log page and the address of the next log page in the linked list. Signed-off-by

[RFC v2 26/83] Add inode_map to track inuse inodes.

From: Andiry Xu NOVA uses per-CPU inode map to track inuse inodes. It works in the same way as the allocator, the only difference is that inode map tracks in-use inodes, while free list contains free ranges. NOVA always try to allocate the first available inode number. Signed-off-by: Andiry Xu

[RFC v2 24/83] Initialize and allocate inode table.

From: Andiry Xu Inode table is a singly linked list of 2MB pages. Each CPU has one inode table with initial size 2MB. The inode table addresses are stored in the INODE_TABLE_START of the pmem range. Signed-off-by: Andiry Xu --- fs/nova/inode.c | 55

[RFC v2 27/83] Save the inode inuse list to pmem upon umount

From: Andiry Xu Signed-off-by: Andiry Xu --- fs/nova/bbuild.c | 48 fs/nova/bbuild.h | 1 + fs/nova/super.c | 1 + 3 files changed, 50 insertions(+) diff --git a/fs/nova/bbuild.c b/fs/nova/bbuild.c index 12a2f11..66053cb 100644 --- a/fs/nova

[RFC v2 25/83] Support get normal inode address and inode table extentsion.

From: Andiry Xu Inodes are assigned to per-CPU inode tables in a round-robin way: If there are four cores, then CPU 0's inode table contains inode 0, inode 4, inode 8, ... CPU 1's inode table contains inode 1, inode 5, inode 9, ... CPU 2's inode table contains inode 2, in

[RFC v2 31/83] Add new vfs inode allocation.

From: Andiry Xu This routine allocates and initializes a new vfs inode, and setup the attributes of corresponding NOVA inode and inode_info. inode operations are missing now. Signed-off-by: Andiry Xu --- fs/nova/inode.c | 144 +++- fs/nova

[RFC v2 28/83] Add NOVA address space operations

From: Andiry Xu direct_IO and writepages support. Signed-off-by: Andiry Xu --- fs/nova/inode.c | 24 fs/nova/inode.h | 1 + 2 files changed, 25 insertions(+) diff --git a/fs/nova/inode.c b/fs/nova/inode.c index 7c10d0e..a30b6aa 100644 --- a/fs/nova/inode.c +++ b/fs

[RFC v2 29/83] Add write_inode and dirty_inode routines.

From: Andiry Xu Signed-off-by: Andiry Xu --- fs/nova/inode.c | 33 + fs/nova/inode.h | 2 ++ fs/nova/super.c | 2 ++ 3 files changed, 37 insertions(+) diff --git a/fs/nova/inode.c b/fs/nova/inode.c index a30b6aa..29d172a 100644 --- a/fs/nova/inode.c +++ b/fs

[RFC v2 32/83] Add log entry definitions.

From: Andiry Xu NOVA appends log entries to the inode log upon metadata change. NOVA has four kinds of log entries: File write entry describes a write to a contiguous range of pmem pages, Dentry describes a file/directory being added or removed from a directory, Setattr entry is used for

[RFC v2 33/83] Inode log and entry printing for debug purpose.

From: Andiry Xu Signed-off-by: Andiry Xu --- fs/nova/nova.h | 3 + fs/nova/stats.c | 234 2 files changed, 237 insertions(+) diff --git a/fs/nova/nova.h b/fs/nova/nova.h index bf4b6ac..03c4991 100644 --- a/fs/nova/nova.h +++ b/fs

[RFC v2 36/83] Journal: Lite journal recovery.

From: Andiry Xu Signed-off-by: Andiry Xu --- fs/nova/journal.c | 55 +++ 1 file changed, 55 insertions(+) diff --git a/fs/nova/journal.c b/fs/nova/journal.c index 75d590f..f31de97 100644 --- a/fs/nova/journal.c +++ b/fs/nova/journal.c

[RFC v2 34/83] Journal: NOVA light weight journal definitions.

From: Andiry Xu NOVA uses per-CPU lite journals to provide fast atomicity guarantees for multi-log appending and multi-word inplace updates. NOVA uses undo journaling. Each journal is a circular buffer of 4KB pmem page. Two pointers, journal_head and journal_tail reside in the reserved journal

[RFC v2 38/83] Journal: NOVA lite journal initialization.

From: Andiry Xu NOVA uses per-CPU spinlock to protect the journals. Lite journal initialization consists of two parts: for a new NOVA instance, hard_init allocates the journal pages. soft_init initializes the locks and performs journal recovery. Signed-off-by: Andiry Xu --- fs/nova/journal.c

[RFC v2 30/83] New NOVA inode allocation.

From: Andiry Xu Allocate the new inode in a round-robin way. Extend the inode table if needed. Signed-off-by: Andiry Xu --- fs/nova/inode.c | 40 fs/nova/inode.h | 1 + 2 files changed, 41 insertions(+) diff --git a/fs/nova/inode.c b/fs/nova/inode.c

[RFC v2 39/83] Log operation: dentry append.

From: Andiry Xu NOVA performs atomic log appending by first appending the entry to the tail of the log, and then atomically update the log tail pointer. Signed-off-by: Andiry Xu --- fs/nova/log.c | 162 ++ fs/nova/log.h | 4 ++ 2 files

[RFC v2 43/83] Log operation: in-place update log entry

From: Andiry Xu To in-place update a log entry, NOVA starts a lite transaction to journal the log entry, then performs update and commits the transaction. Signed-off-by: Andiry Xu --- fs/nova/inode.h | 12 fs/nova/log.c | 183

[RFC v2 35/83] Journal: Lite journal helper routines.

From: Andiry Xu Signed-off-by: Andiry Xu --- fs/nova/Makefile | 2 +- fs/nova/journal.c | 108 ++ 2 files changed, 109 insertions(+), 1 deletion(-) create mode 100644 fs/nova/journal.c diff --git a/fs/nova/Makefile b/fs/nova/Makefile

[RFC v2 37/83] Journal: Lite journal create and commit.

From: Andiry Xu NOVA uses lite journal to perform light weight transaction. Instead of journaling metadata/data changes directly, NOVA first append updates to each inode's log, and then journal the log tail pointers to make sure all the logs are updated atomically. For inode creatio

[RFC v2 40/83] Log operation: file write entry append.

From: Andiry Xu NOVA performs writes by appending file write entries to the log. A file write entry is the metadata of a write operation, and contains pointers to the data blocks. A single write operation may append multiple file write entries to the log, if the allocator cannot provide enough

[RFC v2 41/83] Log operation: setattr entry append

From: Andiry Xu NOVA appends a setattr entry to the log upon inode modification operations: set size, chmod, etc. Signed-off-by: Andiry Xu --- fs/nova/log.c | 64 +++ 1 file changed, 64 insertions(+) diff --git a/fs/nova/log.c b/fs/nova

[RFC v2 44/83] Log operation: invalidate log entries

From: Andiry Xu After new log entries are appended to the log, old log entries can be marked invalid to faciliate garbage collection. Signed-off-by: Andiry Xu --- fs/nova/log.c | 160 + fs/nova/log.h | 4 ++ fs/nova/nova.h | 12

[RFC v2 42/83] Log operation: link change append.

From: Andiry Xu NOVA appends link change entries to atomically update link count and ctime. This occurs in link, unlink and rmdir. Signed-off-by: Andiry Xu --- fs/nova/log.c | 52 fs/nova/log.h | 3 +++ 2 files changed, 55 insertions

[RFC v2 47/83] Dir: Add initial dentries when initializing a directory inode log.

From: Andiry Xu For root directory and newly created directory via mkdir(), we append . and .. dentries to the directory inode log. Signed-off-by: Andiry Xu --- fs/nova/dir.c | 82 + fs/nova/nova.h | 2 ++ fs/nova/super.c | 5

[RFC v2 46/83] Dir: Add Directory radix tree insert/remove methods.

From: Andiry Xu NOVA uses Hash to quickly locate dentry in the directory inode log. The key is the hash of the filename, the value is the dentry. Currently hash collision is ignored, and the radix tree may occupy large memory space with huge directories. Considering replacing it in the future

[RFC v2 50/83] Inode: Add nova_evict_inode.

From: Andiry Xu If the inode still have links, release the DRAM resource (radix tree, etc). Otherwise reclaim data pages and log pages. Signed-off-by: Andiry Xu --- fs/nova/inode.c | 257 +++- fs/nova/inode.h | 5 ++ fs/nova/log.h | 7

[RFC v2 45/83] Log operation: file inode log lookup and assign

From: Andiry Xu After NOVA appends file write entry to commit new writes, it updates the file offset radix tree, finds the old entries (if overwrite) and reclaims the stale data blocks. Signed-off-by: Andiry Xu --- fs/nova/log.c | 108

[RFC v2 49/83] Dir: Append create/remove dentry.

From: Andiry Xu NOVA adds or removes a directory/file by appending a dentry to the parent directory's log. Dentry contains filename and inode number. A positive inode number indicates a create(valid) dentry, and a dentry with inode number zero is a remove dentry. NOVA can also inplace upd

[RFC v2 51/83] Rebuild: directory inode.

From: Andiry Xu When vfs issues a read inode command, or when the inode is newly allocated, walk through the inode log to rebuild inode information and the radix tree. Signed-off-by: Andiry Xu --- fs/nova/inode.h | 15 +++ fs/nova/nova.h| 21 fs/nova/rebuild.c | 329

[RFC v2 52/83] Rebuild: file inode.

From: Andiry Xu Rebuild file inode metadata and radix tree on read_inode. Signed-off-by: Andiry Xu --- fs/nova/log.h | 4 ++ fs/nova/rebuild.c | 124 ++ 2 files changed, 128 insertions(+) diff --git a/fs/nova/log.h b/fs/nova/log.h

[RFC v2 55/83] Namei: mkdir

From: Andiry Xu NOVA mkdir is similar to create. The difference is NOVA will allocate log page for the newly created directory, and append init dentries. Signed-off-by: Andiry Xu --- fs/nova/namei.c | 74 + 1 file changed, 74 insertions

[RFC v2 58/83] Namei: rename

From: Andiry Xu Rename is the most cpmplex namei operation. The target dir may be different from the source dir, and the target inode may exist. Rename involves up to four inodes, and NOVA uses rename transation to atomically update all the affected inodes. Signed-off-by: Andiry Xu --- fs

[RFC v2 57/83] Namei: rmdir

From: Andiry Xu Similar to unlink. Signed-off-by: Andiry Xu --- fs/nova/namei.c | 105 1 file changed, 105 insertions(+) diff --git a/fs/nova/namei.c b/fs/nova/namei.c index 360d716..4bf6396 100644 --- a/fs/nova/namei.c +++ b/fs/nova

[RFC v2 56/83] Namei: link and unlink.

From: Andiry Xu For link change operations, NOVA appends a link change entry to the affected inode's log, and uses lite transaction to atomically commit changes to multiple logs. Signed-off-by: Andiry Xu --- fs/nova/namei.c | 159 ++

[RFC v2 60/83] Add special inode operations.

From: Andiry Xu Signed-off-by: Andiry Xu --- fs/nova/inode.c | 2 ++ fs/nova/namei.c | 5 + fs/nova/nova.h | 1 + 3 files changed, 8 insertions(+) diff --git a/fs/nova/inode.c b/fs/nova/inode.c index 2092a55..0e9ab4b 100644 --- a/fs/nova/inode.c +++ b/fs/nova/inode.c @@ -239,6 +239,7

[RFC v2 59/83] Namei: setattr

From: Andiry Xu Add notify_change for setattr operations. Truncate the file blocks if the file is shrunk. Signed-off-by: Andiry Xu --- fs/nova/inode.c | 180 fs/nova/inode.h | 1 + fs/nova/namei.c | 2 + 3 files changed, 183

[RFC v2 54/83] Namei: create and mknod.

From: Andiry Xu NOVA allocates and initializes a new inode, and appends a dentry to the directory's log. Then NOVA creates a transaction to commit both changes atomically: update the directory log tail pointer and validate the new inode. Signed-off-by: Andiry Xu --- fs/nova/namei.c

[RFC v2 62/83] File: getattr and file inode operations

From: Andiry Xu Signed-off-by: Andiry Xu --- fs/nova/Makefile | 2 +- fs/nova/file.c | 31 +++ fs/nova/inode.c | 25 + fs/nova/inode.h | 2 ++ fs/nova/nova.h | 3 +++ 5 files changed, 62 insertions(+), 1 deletion(-) create mode

[RFC v2 61/83] Super: Add nova_export_ops.

From: Andiry Xu Signed-off-by: Andiry Xu --- fs/nova/super.c | 48 1 file changed, 48 insertions(+) diff --git a/fs/nova/super.c b/fs/nova/super.c index daf3270..0847e57 100644 --- a/fs/nova/super.c +++ b/fs/nova/super.c @@ -51,6 +51,7

[RFC v2 53/83] Namei: lookup.

From: Andiry Xu NOVA lookup the inode number by searching the radix tree with the filename hash value and locating the corresponding dentry on the log. Signed-off-by: Andiry Xu --- fs/nova/Makefile | 3 +- fs/nova/inode.c | 2 ++ fs/nova/namei.c | 97

[RFC v2 63/83] File operation: llseek.

From: Andiry Xu Search the file radix tree to find hold or data. Signed-off-by: Andiry Xu --- fs/nova/file.c | 47 +++ fs/nova/inode.c | 113 fs/nova/inode.h | 1 + fs/nova/nova.h | 1 + 4 files changed, 162

[RFC v2 66/83] Super: Add file write item cache.

From: Andiry Xu nova_file_write_item combines a file write item with a list head. NOVA uses a linked list of file write items to describe a write operation. Signed-off-by: Andiry Xu --- fs/nova/super.c | 43 ++- fs/nova/super.h | 3 +++ 2 files changed

[RFC v2 64/83] File operation: open, fsync, flush.

From: Andiry Xu NOVA persists file metadata and data before returning to the user space. Hence, fsync is a no-op if the file is not mmaped. Signed-off-by: Andiry Xu --- fs/nova/file.c | 50 ++ 1 file changed, 50 insertions(+) diff --git a/fs

[RFC v2 67/83] Dax: commit list of file write items to log.

From: Andiry Xu Given a list of file write items, NOVA commits them by appending each file write entry to the log, and then updates the radix tree to point to these new entries, and updates log tail pointer to commit all the writes atomically. If the items are allocated on heap, free them on

[RFC v2 65/83] File operation: read.

From: Andiry Xu NOVA is a DAX file system and does not use page cache. For read, NOVA looks up the file write entry by searching the radix tree, and copies data from pmem pages to user buffer directly. Signed-off-by: Andiry Xu --- fs/nova/file.c | 144

[RFC v2 68/83] File operation: copy-on-write write.

From: Andiry Xu If the file is not mmaped, NOVA performs copy-on-write. The CoW is composed of parts: 1. Allocate contiguous data pages. 2. Copy data from user buffer to the data pages. If the write is not aligned to page size, also copy data from existing pmem pages. 3. Allocate and

[RFC v2 70/83] File operation: Inplace write.

From: Andiry Xu If the user specifies inplace updates, or the file is mmaped, NOVA performs inplace writes. The trick is dax page fault can occur concurrently with inplace writes, and allocate new blocks. Also, inplace write memcpy may trigger page fault (xfstests 248). Since page fault may

[RFC v2 69/83] Super: Add module param inplace_data_updates.

From: Andiry Xu Provide inplace data updates option if people prefer inplace updates to copy-on-write. Signed-off-by: Andiry Xu --- fs/nova/nova.h | 1 + fs/nova/super.c | 7 ++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/fs/nova/nova.h b/fs/nova/nova.h index 1c2205e

[RFC v2 71/83] Symlink support.

From: Andiry Xu NOVA alloates two blocks for symlink inode: One for inode log, and the other one is a data block, storing symname. Signed-off-by: Andiry Xu --- fs/nova/Makefile | 2 +- fs/nova/inode.c | 2 + fs/nova/namei.c | 70 fs/nova/nova.h| 5

[RFC v2 73/83] Dax: Add iomap operations.

From: Andiry Xu The key of iomap is dax_get_blocks(). It first takes the read lock and lookup the block; if the block is missing, it takes write lock, check again and allocate the new block if needed. Signed-off-by: Andiry Xu --- fs/nova/dax.c | 184

[RFC v2 72/83] File operation: fallocate.

From: Andiry Xu Fallocate works similar as writes, allocating zeroed blocked for the holes in the request region. Signed-off-by: Andiry Xu --- fs/nova/file.c | 148 + fs/nova/nova.h | 5 ++ 2 files changed, 153 insertions(+) diff

[RFC v2 75/83] File operation: read/write iter.

From: Andiry Xu They use the iomap framework to do read/write. Due to software overheads they are slower than dax read/write. Signed-off-by: Andiry Xu --- fs/nova/file.c | 65 ++ 1 file changed, 65 insertions(+) diff --git a/fs/nova

[RFC v2 74/83] File operation: Mmap.

From: Andiry Xu NOVA uses the iomap framework to support mmap operation. Currently it does not support huge page mmap. Signed-off-by: Andiry Xu --- fs/nova/dax.c | 53 + fs/nova/file.c | 25 + fs/nova/nova.h | 1

[RFC v2 76/83] Ioctl support.

From: Andiry Xu NOVA appends link change entry to the inode log to implement SETFLAGS and SETVERSION. Signed-off-by: Andiry Xu --- fs/nova/Makefile | 4 +- fs/nova/dir.c| 4 ++ fs/nova/file.c | 4 ++ fs/nova/inode.h | 2 + fs/nova/ioctl.c | 184

[RFC v2 80/83] Failure recovery: bitmap operations.

From: Andiry Xu Upon system failure, NOVA needs to scan all the inode logs to rebuild the allocator. During the scanning, NOVA stores allocated log/data pages in a bitmap, and uses the bitmap to rebuild the allocator once scan finishes. Signed-off-by: Andiry Xu --- fs/nova/bbuild.c | 252

[RFC v2 79/83] Normal recovery.

From: Andiry Xu Upon umount, NOVA stores the allocator information and the inuse inode list in reserved inodes. During remount, NOVA reads these information and rebuild the allocator and inuse inode list DRAM data structures. Signed-off-by: Andiry Xu --- fs/nova/bbuild.c | 266

[RFC v2 83/83] Sysfs support.

From: Andiry Xu Sysfs support allows user to get/post information of running NOVA instance. After mount, NOVA creates four entries under proc directory /proc/fs/nova/pmem#/: timing_statsIO_statsallocator gc Show NOVA file operation timing statistics: cat /proc/fs/NOVA/pmem

[RFC v2 81/83] Failure recovery: Inode pages recovery routines.

From: Andiry Xu For each inode, NOVA traverses the inode log and records the pages allocated in the bitmap. For directory inode, NOVA only set the log pages. For file and symlink inodes, NOVA needs to set the data pages. NOVA divides the file into 1GB zones, and records the pages fall into the

[RFC v2 82/83] Failure recovery: Per-CPU recovery.

From: Andiry Xu NOVA starts a recovery thread on each CPU, and scans all the inodes in a parallel way. It recovers the inode inuse list during the scan as well. Signed-off-by: Andiry Xu --- fs/nova/bbuild.c | 396 +++ 1 file changed, 396

[RFC v2 77/83] GC: Fast garbage collection.

From: Andiry Xu NOVA cleans and compacts the log when the log is full. The log is a linked list of 4KB pmem pages, and NOVA performs fast garbage collection by deleting dead log pages (all the entries are invalid) from the linked list. Example: I = Invalid, V = Valid VIIV -> ->

[RFC v2 78/83] GC: Thorough garbage collection.

From: Andiry Xu After fast gc, if the valid log entries still account for less than the half of the log size, NOVA starts thorough garbage collection, allocates a new log, copies the live log entries to it, and switches to the new log atomically. The radix tree needs to be updated to point to

[RFC v2 48/83] Dir: Readdir operation.

From: Andiry Xu NOVA reads the directory by traversing the log and reports the valid dentries. Valid dentris have inode number greater than zero, meaning it's a create dentry. Signed-off-by: Andiry Xu --- fs/nova/dir.c | 153 fs

[RFC v2 01/83] Introduction and documentation of NOVA filesystem.

From: Andiry Xu NOVA is a log-structured file system tailored for byte-addressable non-volatile memories. It was designed and developed at the Non-Volatile Systems Laboratory in the Computer Science and Engineering Department at the University of California, San Diego. Its primary authors are

[RFC v2 00/83] NOVA: a new file system for persistent memory

From: Andiry Xu This is the second version of RFC patch series that impements NOVA (NOn-Volatile memory Accelerated file system), a new file system built for PMEM. NOVA's goal is to provide a high performance, production-ready file system tailored for byte-addressable non-volatile mem

[RFC v2 10/83] Add superblock integrity check.

From: Andiry Xu Repair broken primary superblock with redundant superblock. Signed-off-by: Andiry Xu --- fs/nova/super.c | 102 1 file changed, 102 insertions(+) diff --git a/fs/nova/super.c b/fs/nova/super.c index 552fe5d..e0e38ab

Filebench failure on ramdisk with Ext4-DAX

Hi, I am running into failures when run filebench on ramdisk(/dev/ram0) with Ext4-DAX. The kernel version is 4.0, and I also verified it occurs on 4.2-rc1. The issue reproduction steps: // Set ramdisk size to 2GB # mkfs.ext4 /dev/ram0 # mount -o dax /dev/ram0 /mnt/ramdisk # filebench filebench>

  1   2   >