Re: [PATCH] erofs: move erofs out of staging

2019-08-19 Thread Qu Wenruo
[...] >>> I have made a simple fuzzer to inject messy in inode metadata, >>> dir data, compressed indexes and super block, >>> https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/commit/?h=experimental-fuzzer >>> >>> I am testing with some given dirs and the following script. >>>

Re: [PATCH] erofs: move erofs out of staging

2019-08-19 Thread Qu Wenruo
On 2019/8/20 上午10:24, Chao Yu wrote: > On 2019/8/20 8:55, Qu Wenruo wrote: >> [...] >>>>> I have made a simple fuzzer to inject messy in inode metadata, >>>>> dir data, compressed indexes and super block, >>>>> https://git.kernel.org/pu

Re: [PATCH] erofs: move erofs out of staging

2019-08-19 Thread Qu Wenruo
[...] >> The same tool exists for btrfs, although lacks the write ability, but >> that dump is more comprehensive and a great tool to learn the on-disk >> format. >> >> >> And for the fuzzing defending part, just a few kernel releases ago, >> there is none for btrfs, and now we have a full static v

Re: [PATCH] erofs: move erofs out of staging

2019-08-20 Thread Qu Wenruo
[...] > > Yeah, it looks like we need searching more levels mapping to find the final > physical block address of inode/node/data in btrfs. > > IMO, in a little lazy way, we can reform and reuse existed function in > btrfs-progs which can find the mapping info of inode/node/data according to > sp

Re: Readahead for compressed data

2021-10-22 Thread Qu Wenruo
On 2021/10/22 17:11, Gao Xiang wrote: On Fri, Oct 22, 2021 at 10:41:27AM +0200, Jan Kara wrote: On Thu 21-10-21 21:04:45, Phillip Susi wrote: Matthew Wilcox writes: As far as I can tell, the following filesystems support compressed data: bcachefs, btrfs, erofs, ntfs, squashfs, zisofs I

Re: Readahead for compressed data

2021-10-22 Thread Qu Wenruo
On 2021/10/22 17:54, Gao Xiang wrote: On Fri, Oct 22, 2021 at 05:39:29PM +0800, Gao Xiang wrote: Hi Qu, On Fri, Oct 22, 2021 at 05:22:28PM +0800, Qu Wenruo wrote: On 2021/10/22 17:11, Gao Xiang wrote: On Fri, Oct 22, 2021 at 10:41:27AM +0200, Jan Kara wrote: On Thu 21-10-21 21:04:45

Re: [PATCH RFC 2/8] fs: always get the file size in _fs_read()

2022-06-28 Thread Qu Wenruo
On 2022/6/28 20:36, Huang Jianan wrote: Hi, wenruo, 在 2022/6/28 15:28, Qu Wenruo 写道: For _fs_read(), @len == 0 means we read the whole file. And we just pass @len == 0 to make each filesystem to handle it. In fact we have info->size() call to properly get the filesize. So we can not o

Re: [PATCH 0/8] u-boot: fs: add generic unaligned read handling

2022-06-28 Thread Qu Wenruo
On 2022/6/28 22:17, Tom Rini wrote: On Tue, Jun 28, 2022 at 03:28:00PM +0800, Qu Wenruo wrote: [BACKGROUND] Unlike FUSE/Kernel which always pass aligned read range, U-boot fs code just pass the request range to underlying fses. Under most case, this works fine, as U-boot only really needs

Re: [PATCH 1/8] fs: fat: unexport file_fat_read_at()

2022-07-25 Thread Qu Wenruo
On 2022/7/26 06:28, Tom Rini wrote: On Wed, Jun 29, 2022 at 07:38:22PM +0800, Qu Wenruo wrote: That function is only utilized inside fat driver, unexport it. Signed-off-by: Qu Wenruo The series has a fails to build on nokia_rx51: https://source.denx.de/u-boot/u-boot/-/jobs/471877#L483

Re: [PATCH v2 1/8] fs: fat: unexport file_fat_read_at()

2022-08-05 Thread Qu Wenruo
On 2022/8/6 05:14, Tom Rini wrote: On Tue, Jul 26, 2022 at 01:22:09PM +0800, Qu Wenruo wrote: That function is only utilized inside fat driver, unexport it. Signed-off-by: Qu Wenruo Unfortunately, the series fails CI: https://source.denx.de/u-boot/u-boot/-/jobs/478838 OK, it's

Re: [PATCH v2] mm: alloc_pages_bulk: remove assumption of populating only NULL elements

2025-03-04 Thread Qu Wenruo
在 2025/2/28 20:14, Yunsheng Lin 写道: As mentioned in [1], it seems odd to check NULL elements in the middle of page bulk allocating, and it seems caller can do a better job of bulk allocating pages into a whole array sequentially without checking NULL elements first before doing the page bulk a

Re: Compressed files & the page cache

2025-07-15 Thread Qu Wenruo
在 2025/7/16 10:46, Gao Xiang 写道: ... There's some discrepancy between filesystems whether you need scratch space for decompression.  Some filesystems read the compressed data into the pagecache and decompress in-place, while other filesystems read the compressed data into scratch pages an

Re: Compressed files & the page cache

2025-07-15 Thread Qu Wenruo
在 2025/7/16 06:10, Matthew Wilcox 写道: I've started looking at how the page cache can help filesystems handle compressed data better. Feedback would be appreciated! I'll probably say a few things which are obvious to anyone who knows how compressed files work, but I'm trying to be explicit ab

Re: Compressed files & the page cache

2025-07-21 Thread Qu Wenruo
在 2025/7/21 19:55, Jan Kara 写道: On Mon 21-07-25 11:14:02, Gao Xiang wrote: Hi Barry, On 2025/7/21 09:02, Barry Song wrote: On Wed, Jul 16, 2025 at 8:28 AM Gao Xiang wrote: [...] Given the difficulty of allocating large folios, it's always a good idea to have order-0 as a fallback. While

[PATCH 0/8] u-boot: fs: add generic unaligned read handling

2022-06-28 Thread Qu Wenruo via Linux-erofs
work for stat() call. Extra testing/feedback is always appreciated. Qu Wenruo (8): fs: fat: unexport file_fat_read_at() fs: always get the file size in _fs_read() fs: btrfs: move the unaligned read code to _fs_read() for btrfs fs: ext4: rely on _fs_read() to pass block aligned

[PATCH RFC 2/8] fs: always get the file size in _fs_read()

2022-06-28 Thread Qu Wenruo via Linux-erofs
n (e.g. @len > file_size) in advance or truncate @len. This behavior also allows us to handle unaligned better in the incoming patches. Signed-off-by: Qu Wenruo --- fs/fs.c | 25 + 1 file changed, 21 insertions(+), 4 deletions(-) diff --git a/fs/fs.c b/fs/f

[PATCH RFC 1/8] fs: fat: unexport file_fat_read_at()

2022-06-28 Thread Qu Wenruo via Linux-erofs
That function is only utilized inside fat driver, unexport it. Signed-off-by: Qu Wenruo --- fs/fat/fat.c | 4 ++-- include/fat.h | 2 -- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/fs/fat/fat.c b/fs/fat/fat.c index df9ea2c028fc..dcceccbcee0a 100644 --- a/fs/fat/fat.c +++ b

[PATCH RFC 3/8] fs: btrfs: move the unaligned read code to _fs_read() for btrfs

2022-06-28 Thread Qu Wenruo via Linux-erofs
+ @len is, then copy the requested range in the buffer. This has been tested with a proper randomly populated btrfs file, then tried in sandbox mode with different aligned and unaligned range and compare the output with md5sum. Cc: Marek Behun Cc: linux-bt...@vger.kernel.org Signed-off-by: Qu

[PATCH RFC 4/8] fs: ext4: rely on _fs_read() to pass block aligned range into ext4fs_read_file()

2022-06-28 Thread Qu Wenruo via Linux-erofs
lp to clean them up is appreciated. Cc: Tom Rini Signed-off-by: Qu Wenruo --- fs/ext4/ext4fs.c | 11 +++ fs/fs.c | 2 +- include/ext4fs.h | 1 + 3 files changed, 13 insertions(+), 1 deletion(-) diff --git a/fs/ext4/ext4fs.c b/fs/ext4/ext4fs.c index 4c89152ce4ad..be2680994d8b 1

[PATCH RFC 5/8] fs: fat: rely on higher layer to get block aligned read range

2022-06-28 Thread Qu Wenruo via Linux-erofs
Just implement fat_get_blocksize() for fat, so that fat_read_file() always get a block aligned read range. Unfortunately I'm not experienced enough to cleanup the fat code, thus further cleanup is appreciated. Cc: Tom Rini Signed-off-by: Qu Wenruo --- fs/fat/fat.c | 13 + fs

[PATCH RFC 6/8] fs: sandboxfs: add sandbox_fs_get_blocksize()

2022-06-28 Thread Qu Wenruo via Linux-erofs
ly to make testing of _fs_read() much easier. Cc: Simon Glass Signed-off-by: Qu Wenruo --- arch/sandbox/cpu/os.c | 11 +++ fs/fs.c| 2 +- fs/sandbox/sandboxfs.c | 14 ++ include/os.h | 8 include/sandboxfs.h| 1 + 5 files changed, 35

[PATCH RFC 7/8] fs: ubifs: rely on higher layer to do unaligned read

2022-06-28 Thread Qu Wenruo via Linux-erofs
Currently ubifs doesn't support unaligned read offset, thanks to the recent _fs_read() work to handle unaligned read, we only need to implement ubifs_get_blocksize() to take advantage of it. Now ubifs can do unaligned read without any problem. Signed-off-by: Qu Wenruo --- fs

[PATCH RFC 8/8] fs: erofs: add unaligned read range handling

2022-06-28 Thread Qu Wenruo via Linux-erofs
ad support. Cc: Huang Jianan Cc: linux-erofs@lists.ozlabs.org Signed-off-by: Qu Wenruo --- fs/erofs/internal.h | 1 + fs/erofs/super.c| 6 ++ fs/fs.c | 2 +- include/erofs.h | 1 + 4 files changed, 9 insertions(+), 1 deletion(-) diff --git a/fs/erofs/internal.h b/fs/erofs/i

[PATCH 1/8] fs: fat: unexport file_fat_read_at()

2022-06-29 Thread Qu Wenruo via Linux-erofs
That function is only utilized inside fat driver, unexport it. Signed-off-by: Qu Wenruo --- fs/fat/fat.c | 4 ++-- include/fat.h | 2 -- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/fs/fat/fat.c b/fs/fat/fat.c index df9ea2c028fc..dcceccbcee0a 100644 --- a/fs/fat/fat.c +++ b

[PATCH 2/8] fs: btrfs: fix a bug which no data get read if the length is not 0

2022-06-29 Thread Qu Wenruo via Linux-erofs
Fix it by just always do the file size check. In fact btrfs_size() always follow soft link, thus it will return the real file size correctly. Signed-off-by: Qu Wenruo --- fs/btrfs/btrfs.c | 17 + 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/btrfs.c b/fs

[PATCH 0/8] U-boot: fs: add generic unaligned read offset handling

2022-06-29 Thread Qu Wenruo via Linux-erofs
ned offset support. - Semihostfs - Sandboxfs They all use read() directly, no need to do alignment check at all. Extra testing/feedback is always appreciated. Qu Wenruo (8): fs: fat: unexport file_fat_read_at() fs: btrfs: fix a bug which no data get read if the length is not 0 fs: btrfs: fix a crash

[PATCH 4/8] fs: btrfs: move the unaligned read code to _fs_read() for btrfs

2022-06-29 Thread Qu Wenruo via Linux-erofs
unaligned range and compare the output with md5sum. Cc: Marek Behun Cc: linux-bt...@vger.kernel.org Signed-off-by: Qu Wenruo --- fs/btrfs/btrfs.c | 10 fs/btrfs/inode.c | 89 +++- fs/fs.c | 130 --- include/btrfs.h

[PATCH 5/8] fs: ext4: rely on _fs_read() to handle leading unaligned block read

2022-06-29 Thread Qu Wenruo via Linux-erofs
Just add ext4_get_blocksize() and a new assert() in ext4fs_read_file(). Signed-off-by: Qu Wenruo --- Several cleanup candicate: 1. ext_fs->blksz is never populated, thus it's always 0 We can not easily grab blocksize just like btrfs in this case. Thus we have to go the same cal

[PATCH 3/8] fs: btrfs: fix a crash if specified range is beyond file size

2022-06-29 Thread Qu Wenruo via Linux-erofs
: Read range beyond file size, offset 8192 file size 5120 Failed to load '5m_origin' Signed-off-by: Qu Wenruo --- fs/btrfs/btrfs.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/fs/btrfs/btrfs.c b/fs/btrfs/btrfs.c index 9145727058d4..bf9e1f2f17cf 100644 --- a/fs/btrfs/btrfs.

[PATCH 6/8] fs: fat: rely on higher layer to get block aligned read range

2022-06-29 Thread Qu Wenruo via Linux-erofs
Just implement fat_get_blocksize() for fat, so that fat_read_file() always get a block aligned read range. Unfortunately I'm not experienced enough to cleanup the fat code, thus further cleanup is appreciated. Cc: Tom Rini Signed-off-by: Qu Wenruo --- fs/fat/fat.c | 13 + fs

[PATCH 7/8] fs: ubifs: rely on higher layer to do unaligned read

2022-06-29 Thread Qu Wenruo via Linux-erofs
Currently ubifs doesn't support unaligned read offset, thanks to the recent _fs_read() work to handle unaligned read, we only need to implement ubifs_get_blocksize() to take advantage of it. Now ubifs can do unaligned read without any problem. Signed-off-by: Qu Wenruo --- Unfortunately

[PATCH 8/8] fs: erofs: add unaligned read range handling

2022-06-29 Thread Qu Wenruo via Linux-erofs
ad support. Cc: Huang Jianan Cc: linux-erofs@lists.ozlabs.org Signed-off-by: Qu Wenruo --- fs/erofs/internal.h | 1 + fs/erofs/super.c| 6 ++ fs/fs.c | 2 +- include/erofs.h | 1 + 4 files changed, 9 insertions(+), 1 deletion(-) diff --git a/fs/erofs/internal.h b/fs/erofs/i

Re: [PATCH RFC 6/8] fs: sandboxfs: add sandbox_fs_get_blocksize()

2022-06-30 Thread Qu Wenruo via Linux-erofs
On 2022/6/30 18:06, Simon Glass wrote: On Tue, 28 Jun 2022 at 01:28, Qu Wenruo wrote: This is to make sandboxfs to report blocksize it supports for _fs_read() to handle unaligned read. Unlike all other fses, sandboxfs can handle unaligned read/write without any problem since it's ca

[PATCH v2 2/8] fs: btrfs: fix a bug which no data get read if the length is not 0

2022-07-25 Thread Qu Wenruo via Linux-erofs
Fix it by just always do the file size check. In fact btrfs_size() always follow soft link, thus it will return the real file size correctly. Signed-off-by: Qu Wenruo --- fs/btrfs/btrfs.c | 17 + 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/btrfs.c b/fs

[PATCH v2 0/8] U-boot: fs: add generic unaligned read offset handling

2022-07-25 Thread Qu Wenruo via Linux-erofs
d extra help to add block aligned offset support. - Semihostfs - Sandboxfs They all use read() directly, no need to do alignment check at all. Extra testing/feedback is always appreciated. Qu Wenruo (8): fs: fat: unexport file_fat_read_at() fs: btrfs: fix a bug which no data get read if

[PATCH v2 4/8] fs: btrfs: move the unaligned read code to _fs_read() for btrfs

2022-07-25 Thread Qu Wenruo via Linux-erofs
unaligned range and compare the output with md5sum. Cc: Marek Behun Cc: linux-bt...@vger.kernel.org Signed-off-by: Qu Wenruo --- fs/btrfs/btrfs.c | 10 fs/btrfs/inode.c | 89 +++- fs/fs.c | 130 --- include/btrfs.h

[PATCH v2 1/8] fs: fat: unexport file_fat_read_at()

2022-07-25 Thread Qu Wenruo via Linux-erofs
That function is only utilized inside fat driver, unexport it. Signed-off-by: Qu Wenruo --- fs/fat/fat.c | 4 ++-- include/fat.h | 2 -- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/fs/fat/fat.c b/fs/fat/fat.c index df9ea2c028fc..dcceccbcee0a 100644 --- a/fs/fat/fat.c +++ b

[PATCH v2 3/8] fs: btrfs: fix a crash if specified range is beyond file size

2022-07-25 Thread Qu Wenruo via Linux-erofs
: Read range beyond file size, offset 8192 file size 5120 Failed to load '5m_origin' Signed-off-by: Qu Wenruo --- fs/btrfs/btrfs.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/fs/btrfs/btrfs.c b/fs/btrfs/btrfs.c index 9145727058d4..bf9e1f2f17cf 100644 --- a/fs/btrfs/btrfs.

[PATCH v2 5/8] fs: ext4: rely on _fs_read() to handle leading unaligned block read

2022-07-25 Thread Qu Wenruo via Linux-erofs
Just add ext4_get_blocksize() and a new assert() in ext4fs_read_file(). Signed-off-by: Qu Wenruo --- fs/ext4/ext4fs.c | 22 ++ fs/fs.c | 2 +- include/ext4fs.h | 1 + 3 files changed, 24 insertions(+), 1 deletion(-) diff --git a/fs/ext4/ext4fs.c b/fs/ext4

[PATCH v2 6/8] fs: fat: rely on higher layer to get block aligned read range

2022-07-25 Thread Qu Wenruo via Linux-erofs
Just implement fat_get_blocksize() for fat, so that fat_read_file() always get a block aligned read range. Unfortunately I'm not experienced enough to cleanup the fat code, thus further cleanup is appreciated. Cc: Tom Rini Signed-off-by: Qu Wenruo --- fs/fat/fat.c | 13 + fs

[PATCH v2 7/8] fs: ubifs: rely on higher layer to do unaligned read

2022-07-25 Thread Qu Wenruo via Linux-erofs
Currently ubifs doesn't support unaligned read offset, thanks to the recent _fs_read() work to handle unaligned read, we only need to implement ubifs_get_blocksize() to take advantage of it. Now ubifs can do unaligned read without any problem. Signed-off-by: Qu Wenruo --- fs

[PATCH v2 8/8] fs: erofs: add unaligned read range handling

2022-07-25 Thread Qu Wenruo via Linux-erofs
ad support. Cc: Huang Jianan Cc: linux-erofs@lists.ozlabs.org Signed-off-by: Qu Wenruo Reviewed-by: Huang Jianan --- fs/erofs/internal.h | 1 + fs/erofs/super.c| 6 ++ fs/fs.c | 2 +- include/erofs.h | 1 + 4 files changed, 9 insertions(+), 1 deletion(-) diff --git a/fs/ero

[PATCH v3 2/8] fs: btrfs: fix a bug which no data get read if the length is not 0

2022-08-15 Thread Qu Wenruo via Linux-erofs
Fix it by just always do the file size check. In fact btrfs_size() always follow soft link, thus it will return the real file size correctly. Signed-off-by: Qu Wenruo --- fs/btrfs/btrfs.c | 17 + 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/btrfs.c b/fs

[PATCH v3 1/8] fs: fat: unexport file_fat_read_at()

2022-08-15 Thread Qu Wenruo via Linux-erofs
That function is only utilized inside fat driver, unexport it. Signed-off-by: Qu Wenruo --- fs/fat/fat.c | 4 ++-- include/fat.h | 2 -- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/fs/fat/fat.c b/fs/fat/fat.c index df9ea2c028fc..dcceccbcee0a 100644 --- a/fs/fat/fat.c +++ b

[PATCH v3 3/8] fs: btrfs: fix a crash if specified range is beyond file size

2022-08-15 Thread Qu Wenruo via Linux-erofs
: Read range beyond file size, offset 8192 file size 5120 Failed to load '5m_origin' Signed-off-by: Qu Wenruo --- fs/btrfs/btrfs.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/fs/btrfs/btrfs.c b/fs/btrfs/btrfs.c index 309cd595d37d..74a992fa012d 100644 --- a/fs/btrfs/btrfs.

[PATCH v3 0/8] U-boot: fs: add generic unaligned read offset handling

2022-08-15 Thread Qu Wenruo via Linux-erofs
read() calls, really no need to do block alignment check. - Enhanced blocksize check Ensure the returned blocksize is not only non-error, but also non-zero. Qu Wenruo (8): fs: fat: unexport file_fat_read_at() fs: btrfs: fix a bug which no data get read if the length is not 0 fs:

[PATCH v3 4/8] fs: btrfs: move the unaligned read code to _fs_read() for btrfs

2022-08-15 Thread Qu Wenruo via Linux-erofs
unaligned range and compare the output with md5sum. Cc: Marek Behun Cc: linux-bt...@vger.kernel.org Signed-off-by: Qu Wenruo --- fs/btrfs/btrfs.c | 10 fs/btrfs/inode.c | 89 +++- fs/fs.c | 130 --- include/btrfs.h

[PATCH v3 5/8] fs: ext4: rely on _fs_read() to handle leading unaligned block read

2022-08-15 Thread Qu Wenruo via Linux-erofs
Just add ext4_get_blocksize() and a new assert() in ext4fs_read_file(). Signed-off-by: Qu Wenruo --- Several cleanup candicate: 1. ext_fs->blksz is never populated, thus it's always 0 We can not easily grab blocksize just like btrfs in this case. Thus we have to go the same cal

[PATCH v3 6/8] fs: fat: rely on higher layer to get block aligned read range

2022-08-15 Thread Qu Wenruo via Linux-erofs
Just implement fat_get_blocksize() for fat, so that fat_read_file() always get a block aligned read range. Unfortunately I'm not experienced enough to cleanup the fat code, thus further cleanup is appreciated. Cc: Tom Rini Signed-off-by: Qu Wenruo --- fs/fat/fat.c | 13 + fs

[PATCH v3 7/8] fs: ubifs: rely on higher layer to do unaligned read

2022-08-15 Thread Qu Wenruo via Linux-erofs
Currently ubifs doesn't support unaligned read offset, thanks to the recent _fs_read() work to handle unaligned read, we only need to implement ubifs_get_blocksize() to take advantage of it. Now ubifs can do unaligned read without any problem. Signed-off-by: Qu Wenruo --- Unfortunately

[PATCH v3 8/8] fs: erofs: add unaligned read range handling

2022-08-15 Thread Qu Wenruo via Linux-erofs
ad support. Cc: Huang Jianan Cc: linux-erofs@lists.ozlabs.org Signed-off-by: Qu Wenruo --- fs/erofs/internal.h | 1 + fs/erofs/super.c| 6 ++ fs/fs.c | 2 +- include/erofs.h | 1 + 4 files changed, 9 insertions(+), 1 deletion(-) diff --git a/fs/erofs/internal.h b/fs/erofs/i