Add a sysfs node to drop compression-related caches, currently used to
drop in-memory pclusters and cached compressed folios.
Signed-off-by: Chunhai Guo
---
v2->v3:
- All cached folios should be disconnected before invalidating.
- Only add sysfs node when CONFIG_EROFS_FS_ZIP is enabled.
v1:
h
Add a sysfs node to drop compression-related caches, currently used to
drop in-memory pclusters and compressed folios.
Signed-off-by: Chunhai Guo
---
v1:
https://lore.kernel.org/linux-erofs/fabdfe9f-9293-45c2-8cf2-3d86c248a...@linux.alibaba.com
change since v1:
- Change subject as suggested by
Add a sysfs node to drop all compression-related caches, including
pclusters and attached compressed pages.
Signed-off-by: Chunhai Guo
---
Documentation/ABI/testing/sysfs-fs-erofs | 7 +++
fs/erofs/sysfs.c | 11 +++
2 files changed, 18 insertions(+)
diff --g
Once a pcluster is fully decompressed and there are no attached cached
folios, its corresponding `struct z_erofs_pcluster` will be freed. This
will significantly reduce the frequency of calls to erofs_shrink_scan()
and the memory allocated for `struct z_erofs_pcluster`.
The tables below show appro
Once a pcluster is fully decompressed and there are no attached cached
folios, its corresponding `struct z_erofs_pcluster` will be freed. This
will significantly reduce the frequency of calls to erofs_shrink_scan()
and the memory allocated for `struct z_erofs_pcluster`.
The tables below show appro
Clean up the cache when cached decompression strategy is changed to
EROFS_ZIP_CACHE_DISABLED by remount.
Signed-off-by: Chunhai Guo
---
fs/erofs/super.c | 5 +
1 file changed, 5 insertions(+)
diff --git a/fs/erofs/super.c b/fs/erofs/super.c
index 320d586c3896..de2af862e65b 100644
--- a/fs/e
Once a pcluster is fully decompressed and there are no attached cached
folios, its corresponding `struct z_erofs_pcluster` will be freed. This
will significantly reduce the frequency of calls to erofs_shrink_scan()
and the memory allocated for `struct z_erofs_pcluster`.
The tables below show appro
在 2024/11/11 10:28, Gao Xiang 写道:
> Hi Chunhai,
>
> On 2024/11/1 20:42, Chunhai Guo wrote:
>> Add sysfs node to control cached decompression strategy, and all the
>> cache will be cleaned up when the strategy is set to
>> EROFS_ZIP_CACHE_DISABLED.
>>
>> Signed-off-by: Chunhai Guo
> I guess remount
Once a pcluster is fully decompressed and there are no attached cached
folios, its corresponding `struct z_erofs_pcluster` will be freed. This
will significantly reduce the frequency of calls to erofs_shrink_scan()
and the memory allocated for `struct z_erofs_pcluster`.
The tables below show appro
Add sysfs node to control cached decompression strategy, and all the
cache will be cleaned up when the strategy is set to
EROFS_ZIP_CACHE_DISABLED.
Signed-off-by: Chunhai Guo
---
Documentation/ABI/testing/sysfs-fs-erofs | 14 ++
fs/erofs/sysfs.c | 8
Once a pcluster is fully decompressed and there are no attached cached
folios, its corresponding `struct z_erofs_pcluster` will be freed. This
will significantly reduce the frequency of calls to erofs_shrink_scan()
and the memory allocated for `struct z_erofs_pcluster`.
The tables below show appro
> @@ -789,7 +798,16 @@ static int z_erofs_pcluster_begin(struct
> z_erofs_decompress_frontend *fe)
> DBG_BUGON(fe->owned_head == Z_EROFS_PCLUSTER_NIL);
>
> if (!(map->m_flags & EROFS_MAP_META)) {
> - grp = erofs_find_workgroup(sb, blknr);
> + while (1) {
> +
Once a pcluster is fully decompressed and there are no attached cached
pages, its corresponding struct z_erofs_pcluster will be freed. This
will significantly reduce the frequency of calls to erofs_shrink_scan()
and the memory allocated for struct z_erofs_pcluster.
The tables below show approximat
This patch aims to allocate bvpages and short-lived compressed pages
from the reserved pool first.
After applying this patch, there are three benefits.
1. It reduces the page allocation time.
The bvpages and short-lived compressed pages account for about 4% of
the pages allocated from the system
在 2024/8/20 17:27, Gao Xiang 写道:
> Hi Chunhai,
>
> On 2024/8/20 17:25, Chunhai Guo wrote:
>> 在 2024/8/20 16:42, Gao Xiang 写道:
>>> If z_erofs_gbuf_growsize() partially fails on a global buffer due to
>>> memory allocation failure or fault injection (as reported by syzbot
>>> [1]),
>>> new pages nee
在 2024/8/20 16:42, Gao Xiang 写道:
> If z_erofs_gbuf_growsize() partially fails on a global buffer due to
> memory allocation failure or fault injection (as reported by syzbot [1]),
> new pages need to be freed by comparing to the existing pages to avoid
> memory leaks.
>
> However, the old gbuf->pag
在 2024/8/16 1:33, Sandeep Dhavale 写道:
>> I do not have a good number for it now. While the modification is minor
>> and without any side effects. I suggest that we make the change.
>>
>> Thanks,
> Hi Chunhai,
> Even though it may not be easily visible in comprehensive tests like
> app launch in and
在 2024/8/14 14:19, Gao Xiang 写道:
>
> On 2024/8/13 18:28, Chunhai Guo wrote:
>> When the erofs cache decompression is disabled
>> (EROFS_ZIP_CACHE_DISABLED), all pages in pcluster are freed right after
>> decompression. There is no need to cache the pcluster as well and it can
>> be freed.
>>
>> Sig
在 2024/8/14 10:35, Gao Xiang 写道:
> Hi Chunhai,
>
> On 2024/8/13 18:27, Chunhai Guo wrote:
>> As the extra bvec pages are allocated and freed in a short time, it can
> It'd be better to just simplify the subject
> erofs: allocate bvpages from reserved buffer pool first
Got it. I will change it.
>>
When the erofs cache decompression is disabled
(EROFS_ZIP_CACHE_DISABLED), all pages in pcluster are freed right after
decompression. There is no need to cache the pcluster as well and it can
be freed.
Signed-off-by: Chunhai Guo
---
fs/erofs/internal.h | 3 ++-
fs/erofs/zdata.c| 4 ++--
fs
As the extra bvec pages are allocated and freed in a short time, it can
reduce the page allocation time by first allocating pages from the
reserved buffer pool [1].
[1] The reserved buffer pool and its benefits are detailed in
commit 0f6273ab4637 ("erofs: add a reserved buffer pool for lz4
decompr
When erofs cache decompression is disabled (EROFS_ZIP_CACHE_DISABLED),
pages allocated for compressed clusters are freed soon after
decompression. This can reduce the page allocation time by first
allocating pages from the reserved buffer pool [1].
[1] The reserved buffer pool and its benefits are
在 2024/7/22 11:51, Gao Xiang 写道:
> In z_erofs_get_gbuf(), the current task may be migrated to another
> CPU between `z_erofs_gbuf_id()` and `spin_lock(&gbuf->lock)`.
>
> Therefore, z_erofs_put_gbuf() will trigger the following issue
> which was found by stress test:
>
> <2>[772156.434168] kernel BU
This adds a special global buffer pool (in the end) for reserved pages.
Using a reserved pool for LZ4 decompression significantly reduces the
time spent on extra temporary page allocation for the extreme cases in
low memory scenarios.
The table below shows the reduction in time spent on page allo
It will cost more time if compressed buffers are allocated on demand for
low-latency algorithms (like lz4) so EROFS uses per-CPU buffers to keep
compressed data if in-place decompression is unfulfilled. While it is kind
of wasteful of memory for a device with hundreds of CPUs, and only a small
num
在 2024/4/2 17:22, Gao Xiang 写道:
>
> On 2024/4/2 17:27, Chunhai Guo wrote:
>> Let's use alloc_pages_bulk_array() for simplicity and get rid of
>> unnecessary pagepool.
>>
>> Signed-off-by: Chunhai Guo
>> ---
> Would you mind adding a changelog here next time? no
> matter how short the text is.
>
>
Let's use alloc_pages_bulk_array() for simplicity and get rid of
unnecessary pagepool.
Signed-off-by: Chunhai Guo
---
fs/erofs/zutil.c | 67 ++--
1 file changed, 31 insertions(+), 36 deletions(-)
diff --git a/fs/erofs/zutil.c b/fs/erofs/zutil.c
index
This adds a special global buffer pool (in the end) for reserved pages.
Using a reserved pool for LZ4 decompression significantly reduces the
time spent on extra temporary page allocation for the extreme cases in
low memory scenarios.
The table below shows the reduction in time spent on page allo
Let's use alloc_pages_bulk_array() for simplicity and get rid of
unnecessary pagepool.
Signed-off-by: Chunhai Guo
---
fs/erofs/zutil.c | 64 +++-
1 file changed, 30 insertions(+), 34 deletions(-)
diff --git a/fs/erofs/zutil.c b/fs/erofs/zutil.c
index
It will cost more time if compressed buffers are allocated on demand for
low-latency algorithms (like lz4) so EROFS uses per-CPU buffers to keep
compressed data if in-place decompression is unfulfilled. While it is kind
of wasteful of memory for a device with hundreds of CPUs, and only a small
num
Currently, utils.c is only useful if CONFIG_EROFS_FS_ZIP is on.
So let's rename as zutil.c as well as avoid its inclusion if
CONFIG_EROFS_FS_ZIP is explicitly disabled.
Signed-off-by: Chunhai Guo
Suggested-by: Gao Xiang
---
fs/erofs/Makefile | 4 ++--
fs/erofs/{utils.c => zutil.c}
It will cost more time if compressed buffers are allocated on demand for
low-latency algorithms (like lz4) so EROFS uses per-CPU buffers to keep
compressed data if in-place decompression is unfulfilled. While it is kind
of wasteful of memory for a device with hundreds of CPUs, and only a small
num
It will cost more time if compressed buffers are allocated on demand for
low-latency algorithms (like lz4) so EROFS uses per-CPU buffers to keep
compressed data if in-place decompression is unfulfilled. While it is kind
of wasteful of memory for a device with hundreds of CPUs, and only a small
num
Got it. I will send patch v2 as your suggestion.
Thanks,
在 2024/3/22 12:10, Gao Xiang 写道:
>
> On 2024/3/22 11:47, Chunhai Guo wrote:
>> It will cost more time if compressed buffers are allocated on demand for
>> low-latency algorithms (like lz4) so EROFS uses per-CPU buffers to keep
>> compressed
It will cost more time if compressed buffers are allocated on demand for
low-latency algorithms (like lz4) so EROFS uses per-CPU buffers to keep
compressed data if in-place decompression is unfulfilled. While it is
kind of wasteful of memory for a device with hundreds of CPUs, and only
a small num
> -邮件原件-
> 发件人: Gao Xiang
> 发送时间: 2024年1月26日 11:50
> 收件人: Chunhai Guo ; xi...@kernel.org
> 抄送: c...@kernel.org; huy...@coolpad.com; jeffl...@linux.alibaba.com;
> linux-erofs@lists.ozlabs.org
> 主题: Re: [PATCH v2] erofs: relaxed temporary buffers allocation on readahead
>
> [你通常不会收到来自 hsi
On 2024/1/26 10:47, Gao Xiang wrote:
> [你通常不会收到来自 hsiang...@linux.alibaba.com 的电子邮件。请访问
> https://aka.ms/LearnAboutSenderIdentification，以了解这一点为什么很重要]
>
> On 2024/1/26 10:41, Chunhai Guo wrote:
>> On 2024/1/22 15:42, Chunhai Guo wrote:
>>> On 2024/1/22 12:37, Gao Xiang wrote:
[你通常不会收到来自 hsiang
On 2024/1/22 15:42, Chunhai Guo wrote:
> On 2024/1/22 12:37, Gao Xiang wrote:
>> [你通常不会收到来自 hsiang...@linux.alibaba.com 的电子邮件。请访问
>> https://aka.ms/LearnAboutSenderIdentification，以了解这一点为什么很重要]
>>
>> On 2024/1/22 11:49, Chunhai Guo wrote:
>>> On 2024/1/22 10:07, Gao Xiang wrote:
[你通常不会收到来自 hsi
On 2024/1/22 12:37, Gao Xiang wrote:
> [你通常不会收到来自 hsiang...@linux.alibaba.com 的电子邮件。请访问
> https://aka.ms/LearnAboutSenderIdentification，以了解这一点为什么很重要]
>
> On 2024/1/22 11:49, Chunhai Guo wrote:
>> On 2024/1/22 10:07, Gao Xiang wrote:
>>> [你通常不会收到来自 hsiang...@linux.alibaba.com 的电子邮件。请访问
>>> https:/
On 2024/1/22 10:07, Gao Xiang wrote:
> [你通常不会收到来自 hsiang...@linux.alibaba.com 的电子邮件。请访问
> https://aka.ms/LearnAboutSenderIdentification，以了解这一点为什么很重要]
>
> On 2024/1/20 22:55, Chunhai Guo wrote:
>> Even with inplace decompression, sometimes extra temporary buffers are
>> still needed for decompressi
Even with inplace decompression, sometimes extra temporary buffers are
still needed for decompression. In low-memory scenarios, it would be
better to try to allocate with GFP_NOWAIT on readahead first. That can
help reduce the time spent on page allocation under memory pressure.
There is an avera
During decompression, it is better to allocate readahead pages with the
GFP_NOWAIT flag, which can help reduce the time spent on page allocation in
low memory scenarios.
>From the result of multi-app launch benchmarks on ARM64 Android devices
running the 5.15 kernel with an 8-core CPU and 8GB of m
On 2024/1/10 14:45, Chunhai Guo wrote:
> On 2024/1/9 21:08, Gao Xiang wrote:
>> [你通常不会收到来自 hsiang...@linux.alibaba.com 的电子邮件。请访问
>> https://aka.ms/LearnAboutSenderIdentification，以了解这一点为什么很重要]
>>
>> Hi Chunhai,
>>
>> On 2024/1/9 15:41, Chunhai Guo wrote:
>>> Using a global page pool for LZ4 decompr
On 2024/1/9 21:08, Gao Xiang wrote:
> [你通常不会收到来自 hsiang...@linux.alibaba.com 的电子邮件。请访问
> https://aka.ms/LearnAboutSenderIdentification，以了解这一点为什么很重要]
>
> Hi Chunhai,
>
> On 2024/1/9 15:41, Chunhai Guo wrote:
>> Using a global page pool for LZ4 decompression significantly reduces the
>> time spent o
Using a global page pool for LZ4 decompression significantly reduces the
time spent on page allocation in low memory scenarios.
The table below shows the reduction in time spent on page allocation for
LZ4 decompression when using a global page pool. The results were
obtained from multi-app launch
Make erofs_err() and erofs_info() support NULL sb parameter for more
general usage.
Suggested-by: Gao Xiang
Signed-off-by: Chunhai Guo
---
V3 -> V4: remove unnecessary '\n'
V2 -> V3: convert pr_err() to erofs_err() in erofs codebase
V1 -> V2: add erofs_info()
---
fs/erofs/decompressor_deflate.c
Make erofs_err() and erofs_info() support NULL sb parameter for more
general usage.
Suggested-by: Gao Xiang
Signed-off-by: Chunhai Guo
---
V2 -> V3: convert pr_err() to erofs_err() in erofs codebase
V1 -> V2: add erofs_info()
---
fs/erofs/decompressor_deflate.c | 2 +-
fs/erofs/super.c
Make erofs_err() and erofs_info() support NULL sb parameter for more
general usage.
Suggested-by: Gao Xiang
Signed-off-by: Chunhai Guo
---
fs/erofs/super.c | 10 --
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/fs/erofs/super.c b/fs/erofs/super.c
index 3789d6224513..5f60
Make erofs_err() and erofs_info() support NULL sb parameter for more
general usage.
Suggested-by: Gao Xiang
Signed-off-by: Chunhai Guo
---
fs/erofs/super.c | 11 +--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/fs/erofs/super.c b/fs/erofs/super.c
index 3789d6224513..96a
Make erofs_err() support NULL sb parameter for more general usage.
Suggested-by: Gao Xiang
Signed-off-by: Chunhai Guo
---
fs/erofs/super.c | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/fs/erofs/super.c b/fs/erofs/super.c
index 3789d6224513..f20dcfacdcc5 100644
--- a/f
On 2023/12/31 9:09, Gao Xiang wrote:
> [你通常不会收到来自 hsiang...@linux.alibaba.com 的电子邮件。请访问
> https://aka.ms/LearnAboutSenderIdentification，以了解这一点为什么很重要]
>
> On 2023/12/29 12:48, Chunhai Guo wrote:
>>> Hi Chunhai,
>>>
>>> On 2023/12/28 21:00, Chunhai Guo wrote:
Using a global page pool for LZ4 d
> Hi Chunhai,
>
> On 2023/12/28 21:00, Chunhai Guo wrote:
> > Using a global page pool for LZ4 decompression significantly reduces
> > the time spent on page allocation in low memory scenarios.
> >
> > The table below shows the reduction in time spent on page allocation
> > for
> > LZ4 decompres
Using a global page pool for LZ4 decompression significantly reduces the
time spent on page allocation in low memory scenarios.
The table below shows the reduction in time spent on page allocation for
LZ4 decompression when using a global page pool.
The results were obtained from multi-app launch
53 matches
Mail list logo
